Measuring QoE of mobile data applications for network optimization (part 3)

Network operators want to optimize their network for real use-case scenarios with mobile applications, for example when trying to achieve a good benchmarking result. However, the full performance of such an application service is not under the mobile network operator’s control. The entire chain, and not just by the performance of the airlink, determines the quality of experience (QoE).

The chain starts with the third-party server’s performance and includes the actions of third-party companies, such as YouTube or Facebook, and how they are linked to the Internet. This is not under the control of the operator. Of course, the connection of the operator’s core network to the Internet may also influence the final performance.

The operator can solve such issues, but they are often not easily visible when focusing only on the RF parameters. Here, a wider analysis that includes higher layers is required. Finally, the app itself has a large influence on the final performance. The app is usually in close communications with the server at the other end and adjusts to momentary channel states through feedback loops.

We have also to consider that even a simple looking service, such as video streaming or opening a website, initiates much more than just one link to the content server. There are many connections and individual parallel activities to provide advertisements, wrapping information, and reports about user settings and preferences. By far not all of these background activities are visible to nor wanted by the user; however, they are part of the service and consume both data capacity and time.

How to use mobile app testing for optimization

Mobile network optimization needs technical tests with detailed results that reveal the technical parameters where improvements can be made. Consequently, there is no single test that can be used to optimize the network and guarantee the best QoE for the target app test. Instead, an iterative procedure leads to the best results:

• Use the target app test to check the QoE from a real user perspective. This test includes the entire chain that determines the QoE for the users. Here, it is very important to mimic the real use case as closely as possible by using typical file sizes, types, and so on.
• If the QoE is not satisfactory, determine which technical parameter in the network could be optimized. This might require additional technical testing. Weaknesses on the client or server-side cannot be addressed directly, but sometimes a change in the network can reduce their impact.
• After optimization, repeat the app test and compare the results with step 1. If the QoE does not improve, further optimization of the technical parameter in question might not be helpful and you may have reached the optimization limit. The network is “good enough” in this respect.
• Go back to step 2 and determine more optimization points until the app test QoE is satisfactory.

As an example, a mobile network operator could have been rated slow for uploads to Facebook in a benchmarking campaign. Using classical HTTP transfer tests, he finds out that his average HTTP transfer throughput is also slower compared with competitors.

But after another Facebook test in a cell optimized for high throughputs, it becomes clear that the upload duration of a file to Facebook is only marginally faster than before. The average available HTTP transfer throughput was already good enough. A closer look at the results of the Facebook test reveals that the most important factor for the upload speed is not the throughput.

Throughput is not necessarily key for satisfactory QoE.

Instead, it is crucial how fast a third-party server can be accessed from the backbone of the network and if a preference for the high performant Facebook servers is available or not. After making improvements in the third-party server accessibility, the app test QoE is finally satisfactory.

When the general strategy is clear, that both app tests and technical tests are needed to yield the best QoE in the real use-case scenarios, the challenge remains to find out which technical parameters are promising optimization candidates that will, in the end, be reflected in the QoE.

Download the white paper “Insights into the QoE of social media applications on smartphones – a large-scale real field analysis” and learn more about mobile app testing, the test setup, and findings.