Written by Simon Allemann | August 28, 2019
The NPS methodology relies on established metrics, such as success ratio and setup time, that are based on a transparent method. Furthermore, it allows pinpointing weak spots and the underlying root causes of non-optimal performance.
Compare apples to apples
Collecting the necessary data seems like an easy task. However, professional benchmarking has to guarantee the comparison of apples to apples.
It starts with the smartphones and user equipment (UE) respectively used for testing. All of them have to be operated under the same conditions, for example, the same ambient temperature. If one UE gets too hot, it throttles its central processing unit (CPU), having an immediate impact on the UE’s performance. And a significant impact on the scoring of that operator, of course.
On the test level, the controlling software has to ensure that each test starts at the same time. Again, to guarantee equal conditions for all measurement channels. When comparing KPIs used for the calculation of the NPS, each channel must execute the same test at the same time and under the same conditions. For this, the R&S SmartBenchmarker software offers a function called “sync action” in its NPS templates.
The “sync action” software module checks the status of the running test on all involved channels and allows starting the next test only after the previous one has finished. This guarantees the synchronization of all channels.
Network Performance Score templates
Rohde & Schwarz offers NPS templates so that users can set up their NPS campaign easily and without much effort: all the tricky parts are predefined in the templates. Only the hostnames of the server or the URLs of the web pages have to be modified and adapted to the corresponding country or region. Rohde & Schwarz also provides guidelines on what kind of URLs should be used in certain regions/countries.
There are two NPS template types available: one for telephony and one for data services.
The basic structure of the Network Performance Score
The telephony subscore aims at evaluating the experience of the end user regarding telephony services. When calculating the score, the service’s availability, accessibility, retainability, and performance are evaluated. The telephony or call test template includes a double-ended call with speech quality evaluation and the required “sync action”. This call template is ready for use without further modifications.
Network Performance Score for data services
The subscore for data services consists of three contributing areas that address different types of services and characterize different types of requests in a network. The three areas are:
- (plain) data transfer (HTTP)
- video streaming
- HTTP browsing and social media
Video services make up the majority of the data transferred in mobile networks – with YouTube being the most popular streaming service – and must be considered in testing. Besides measuring the pure access times and perceived quality, video tests are very demanding on the network, particularly for ensuring continuous data delivery to avoid outages. Consequently, it is a valuable extension to the plain HTTP data transfer tests, where a continuous transfer is not required and cannot be measured.
For the data services tests, R&S offers two options: a simple setup with one session or a more complex session with three similar interleaving sessions. The data session usually starts with a pre-load ping test followed by a selection of data tests that reflect the behavior of smartphone users today: file download, file upload, browsing, video streaming, and social media/OTT applications.
In the data service template, the user simply has to replace the URLs with the parameters of the corresponding market. After that, the test is ready.
Transparent and objective benchmarking
The NPS delivers a scoring mechanism that allows a very efficient and reliable comparison between operators in a market, and between different measurement campaigns in regions and countries, or before and after the deployment of new technology or software. The transparent structure of the score allows an efficient drill down into the region or service and into the KPIs responsible for a non-optimal overall score.
The smooth integration and support of the NPS in the Rohde & Schwarz benchmarking product portfolio offers customers the right tool for getting more value out of their network.
Find comprehensive information about the NPS on our dedicated web page.
