WhatsApp, Viber, Discord, Facetime, Skype … there’s no denying that people are increasingly choosing OTT apps for their voice calling. With operator-billed mobile voice revenue forecast by industry analysts to decline by 16% over the next four years as P2P voice traffic migrates to OTT telephony apps, mobile operators have a decision to make around OTT testing.
How do you ensure that your own mobile voice services deliver a hi-fidelity experience at least comparable to OTT apps while also understanding the impact of your network on these OTT services and the end user experience that your subscribers are getting as a result?
In our previous blog we looked at the challenge for operators of launching 5G voice services and why a new ML-based, network-centric approach to voice testing is required. In this blog, we’ll take on the other side of the equation – how can operators test and troubleshoot the performance of encrypted OTT telephony running over their networks?
Why is OTT Testing important for encrypted OTT voice services?
The roll-out of 5G networks is driving new services and OTT applications which users expect to perform as flawlessly as everything else they do on their device. Delivering on this expectation of seamless user experience is a significant challenge for operators due to the increased complexity of the network, encryption of services/applications, proprietary codecs/clients/delivery protocols and the constant pressure to minimize network operational costs.
The influence of a specific device type or a specific OTT telephony application is not of significant concern for operators since device vendors and OTT providers ensure the performance of their own products. The challenge for operators is how to understand their network’s impact on voice quality – and this requires a generic OTT approach to testing. Using generic OTT clients moves OTT telephony application testing from an unstable, variable black box solution to a transparent, controllable benchmark.
What is generic OTT Testing for telephony applications?
A generic OTT telephony application uses a generic OTT (gOTT) voice client to represent a network-centric benchmark for OTT telephony testing. gOTT has three main features to compensate for the challenges mentioned above:
- It is independent of the device audio giving a network-centric view
- It provides a stable application version without unknown changes
- It has a transparent client, free of encryption and error concealment schemes, which exposes the reason for all types of audio degradations
Infovista’s research, as well as ongoing standards initiatives (such as ETSI, ITU-T), have proven that if the generic approach closely mimics the real service/OTT application, then the generic approach delivers trustful and representative results. Closely mimicking a real service/OTT application means that the generic approach needs to:
- Trigger the same network resources
- Use the same protocols
- Encode and adapt
as real services and OTT codecs and clients would.
How can you test an OTT telephony application?
Accurate generic OTT testing starts with the benchmark you are using. WhatsApp is one of the most popular native OTT telephony applications with more than 2 billion people in over 180 countries using it. But how do you overcome the fact that the traffic is encrypted? WhatsApp also has the significant advantage of using an open-source codec and jitter buffer, both with many configurable settings.
By conducting a thorough analysis of real WhatsApp application traffic patterns in different network conditions, ranging from very poor to excellent, Infovista’s research team (led by Dr. Irina Cotanis, technology director in the Office of the CTO) was able to adapt these settings, resulting in gOTT telephony closely mimicking the voice quality produced by the most used mobile native OTT telephony application.
In the new “OTT telephony application testing whitepaper”, authored by Dr Irina together with Per Johansson, they set out in full detail the configuration process they went through to design Infovista’s gOTT telephony client. This included:
- Simultaneously analyzing Real-time Transport Protocol (RTP) packets, their corresponding audio samples/sequences, and their quality MOS scoring recorded in good radio quality.
- Comparing gOTT and WhatsApp applications running on the same device to acquire voice quality MOS scoring at the same time stamps
- Developing generic OTT telephony KPIs to be used for testing jitter buffer handling
Why does OTT telephony application testing matter?
The variety and diversity of 4G/5G services and OTT applications combined with increased media encryption and the complexity of mobile access technologies is creating significant technical and cost challenges which call for a generic testing approach. Generic OTT telephony application testing allows operators to invest their efforts in what they can fix and optimize (the network), ensuring customer satisfaction for the minimum operational costs.
In the second whitepaper of this 2-part series, the team presents a data-driven validation of the suitability of a generic OTT telephony application for replacing a native one, looking in more detail into voice quality (MOS scoring), speech path delay, call set-up time, call failure and completion – but more on that in the next blog!
Learn more about
To find out more about OTT telephony application testing, please contact us, or dig into some of the helpful resources below.
More helpful pages:
- Whitepaper: Designing a generic OTT telephony application for accurate quality assessment and troubleshooting of encrypted OTT voice services
- Whitepaper: A data-driven validation of the Infovista TEMS™ generic OTT telephony application vs. a native OTT app
- ebook: Evolving network testing – 12 use cases
- Whitepaper: A new approach for testing voice quality
- Webinar: A Machine Learning Approach for Network Centric Voice Quality Testing
