How to test a mobile network?

How to ensure end-user quality of experience?

Mobile networks are highly complex structures, intricately designed to meet the quality requirements of the network and subscribers equally. To subscribers, the labyrinthine construction of a mobile network, however, appears transparent and straightforward: either they receive the expected service or they do not. Whether this service is making a call, watching a video, or posting on a social media timeline does not matter to them.

Subscribers equate quality of service (QoS) with quality of experience (QoE), regardless of whether a mobile network operator (MNO) or a third-party content provider controls the service. So, where does this leave the MNO who wants to ensure end-user quality of experience and optimize his network and services respectively? What will he need to consider before investing in solutions for mobile network testing?

The mobile network structure today

The mobile network consists of the radio access network (RAN) and the core network; in LTE networks, it is eNodeBs and the enhanced packet core (ePC) respectively. User equipment (UE), such as smartphones, connects to the operator’s mobile network. The mobile network operator controls the servers for own voice and video services. Via the same infrastructure, subscribers also massively enjoy applications from third parties, so-called over-the-top (OTT) media services. Typically, an application client that is located on the smartphone connects to an application server that is out of the MNO’s reach. Consequently, MNOs have no direct control over third-party apps such as YouTube, Spotify, etc. We recommend using different tests to test either network quality or quality of experience.

For more detailed information please watch the video.

Challenges

Network quality: Network capacity and coverage

When measuring network quality, in addition to information from the operator’s OSS, you should also measure network capacity from the user perspective, including geographical dimensions. Knowing the possible data throughput at a certain location helps you identify your network’s optimization needs in terms of urgency and location.

If you want to understand your network’s performance, we suggest using dedicated network performance tests. Excluding third-party components when testing, such as application servers beyond the MNO’s control, will give you a true view of your and your competitor’s mobile network.

Quality of experience: Performance of own or OTT application

Today and even more in future the vast majority of the data is and will be consumed by smartphones and their installed applications.

If you want to know how OTT applications are performing in your network – and with that, how subscribers are using them – then you should use precisely those applications to conduct tests. This type of testing reflects the real behavior of end users and consequently measures true quality of experience (QoE).

Testing of video quality in mobile networks

Testing networks vs. testing applications

Example: Video

At the MNO network level, you need to ensure good coverage and sufficiently high capacity. The easiest way to check this is with the R&S “Network Performance Test”.
If you are offering an own video service, you need to verify its quality using the QualiPoc Android generic video quality test. You can conduct this test under quite controllable conditions.
Finally, you need to evaluate the most popular third-party server/client-based video apps in your network, namely YouTube and others. The dedicated video quality test application in QualiPoc Android will do that for you.

Video quality testing (video MOS acc. ITU-T J343.1)

QualiPoc Android offers an exclusive video quality analysis application. The test evaluates the time to first picture, the visual quality depicted as a video MOS score (acc. ITU-T J343.1), stalling and jerkiness of live streamed content and video-on-demand.

If you want to read more about video quality analysis, check out our video quality testing page.

Blog posts

The true view of your and your competitors’ networks

Hand in hand with the ever increasing demand for more network capacity and the soaring popularity of applications, particularly in real-time, mobile operators in mature markets are also facing a virtually flat subscriber penetration for the next five years. To win new customers, mobile operators are left with no other option than to beat the competition by providing better services and a higher quality of experience from a true end-user perspective.

More information

Visual quality and video MOS based on subjective perception

Video services are very popular and so is their use today in various services and media. YouTube and Netflix are merely the pioneers in video streaming; many other services have followed. Broadcasting stations move more and more content online, videos are shared via social media, and peer-to-peer applications are on the rise; for example, video chat services (e.g. Skype), or the private broadcasting of live captured videos (e.g. Periscope). But what’s their visual quality like?

More information

Optimizing a network’s readiness for video transmission

In previous entries of this blog series about the challenges of mobile video services, we discussed video quality as a perceived metric, how it can be measured, and how quality measuring algorithms work. To conclude the series, this post focuses on how to apply a video quality measure to quantify video services as consumed by subscribers. What are the challenges and what are the results telling you about your network?

More information

Different models for predicting visual quality and related test cases

In this post, we will take a closer look at automated video quality measurement methods, often called objective measurements, which are applied in real field video quality measurements.

More information

Videos

Webinar: Are you really taking care of QoE in your mobile network?

With operators under pressure to deliver ever-higher Quality of Experience (QoE) of services to their subscribers whilst ensuring opex and capex efficiencies, this webinar examines the role of drive testing and benchmarking to provide the reliable end-user insight needed for intelligent network investment decisions. Furthermore, it looks at the factors influencing accuracy and reliability of data acquisition and data analytics and how correct measurement equipment and technique can enable the efficient delivery of better services and higher quality to customers.

More information

Webinar: Why voice quality is still a differentiator in today's mobile networks

The proportion of mobile network traffic represented by voice communications has been steadily decreasing so now it represents around 2% of traffic compared with around 70% for video. However, voice remains the principle service for which most subscribers use their mobile device, and voice quality continues to be a critical factor in how they perceive their operator and OTT applications. This webinar examines the significance of voice in today’s mobile networks and looks at the importance of an integrated metric that quantifies voice quality from an end-user perspective.

More information

Webinar: Enabling smart macro network enhancements

The webinar examines the challenge facing operators to address increasing capacity and performance demands in mobile networks and considers various macro network enhancements such as 4x4 MIMO, carrier aggregation, etc.

More information

User-perceived network performance test

In the video, Arnd Sibila, Technology Marketing Manager, explains how to measure network performance, capacity and coverage from an end-user perspective. Arnd describes the test scope, including all significant parameters.

Overview of video services in mobile networks

In the video, Arnd Sibila, Technology Marketing Manager, explains the increasingly important role of video quality in mobile networks. He discusses the limitations that operators face in controlling video quality, because it depends on network resource availability and allocation.

Testing perceived video quality in mobile networks - part I and II

The video explains how objective methods and algorithms, such as the video MOS J.343.1, are used to measure the subjective nature of video quality in mobile networks. The algorithm J.343.1 combines all video characteristics to obtain a quality value, including coding, compression, freezing, etc.

Documents

Poster: Maximize network quality by harmonized test methods

This poster visualizes how we calculate a single QoE-centric score in order to characterize the status of a mobile network in a region or at country-wide level.

More information