In the first part of a series discussing drive testing of 5G NR networks, we outlined the five most important considerations for 5G NR testing emerged from the technology disruptions which 5G brings to the air interface. In the second, we argued why drive-testing remains relevant to 5G New Radio (NR) testing, despite the increased level of intelligence embedded in the 5G air interface, both network and devices.
We pointed out, however, that the nature as well as the complexity of 5G NR-enabling technologies come with new requirements for drive testing, and thus traditional drive testing as we know it has to change from several points of view.
Continuing our exploration of subjects covered in our detailed white paper, we now look at how Infovista is rethinking TEMS probes.
The main reasons for device-centric 5G NR testing
Four reasons have driven the emergence of device-centric 5G NR testing:
- The extreme performance values need to be evaluated at the device level. Very high user application data rates (e.g. tens of Gb/s) enabled by NR require test scenarios that need to max out the data pipe toward the device in order to evaluate the effective maximum data rate perceived by the user of a specific device. In addition, very low delay (e.g. RTT<1ms) values can come sensitively close to inherited test equipment measurement accuracy, thus opening the risk of misleading measurements results.
- New technology disruptions (e.g. beamforming) that require more information elements (IE), more measurement data, and higher acquisition rates, all experienced and needed for the device to cope. For example, with LTE, a 100ms aggregation time window for RF measurements is generally sufficient to accurately describe an RF field condition, while for NR, less than 20ms can be enough, depending on the number of beams scanned.
- 3D testing scenarios, and consequently 3D results visualization emerged from 3D beamforming antenna technologies, whose benefits and gains vs those of LTE can be quantified only based on their impact on the device.
- The increased intelligence embedded in the network and the device emphasize the need for augmented and predictive, rather than traditional reactive, measurements. This requirement is generally met by defining NR-specific new measurement events and new measurement IE, in addition to the ones provided by the device vendor’s Interface Control Document (ICD), and use those along with ML algorithms to augment device measurements.
Consequently, unlike LTE, 5G NR device-based tools require probes that have adequate and adaptive temporal resolution, allow 3D testing and visualization, and enable predictive rather than reactive test probes.
Temporal resolution and information element datasets
To fit a large number of 5G NR IEs (including all of those that describe beam characteristics), TEMS implemented IEs and measurement events depending on the ICD content, along with an incremental approach to accommodate ICD requirements and updates. In addition, flexibility and scalability is ensured through the possibility of visualizing the information in the mode reports prior to the implementation of an IE.
Regarding the temporal resolution, TEMS is using two different temporal user equipment (UE) resolutions, depending on the tested use case. Such an example is EN-DC testing, for which the synchronization between the two radio access technologies (RATs) needs to be visualized; TEMS Reference Signals Received Power (RSRP) measurement on LTE is reported around every 20ms and contains RSRP for a specific subframe, while RSRP measurement on NR is reported around every 10ms and contains beam RSRP for a specific frame.
Predictive testing probes, augmented measurements and automation
With TEMS, we’re rethinking testing probes on an ongoing basis, moving from reactive to more proactive measurements, and consequently reducing post-processing time, as well as enabling on-device root-cause analysis (RCA) to augment device- based measurements.
Traditionally (Figure 1, which shows the rethinking of TEMS probes with ML/AI and edge computing), field testing probes collect RF logfiles from the device, and measurements are correlated with GPS information and other external data sources (e.g. site information). The collected data enables on-the-spot analysis of a call by playing back the recorded data on the TEMS Investigation drive testing solution, and/or more comprehensive analysis and Root Cause Analysis over a large set of calls in the TEMS Discovery post-processing solution. In addition, the real-time orchestrator, TEMS Director, can stream in real-time the collected data for further analysis and performance reports generation.
- With 5G NR, TEMS engineers are currently working on rethinking field testing probes, including unattended probes, with ML/AI algorithms, which enable the augmentation of TEMS device-based measurements. This is achieved in three ways (Figure 1) First, with ML real-time prediction of performance metrics (such as throughput and latency) in TEMS drive testing solutions (e.g. TEMS Pocket);
- Second, with ML-based RCA in TEMS Director, to be used on large sets of calls (collected logfiles) to both identify networks problems that bear significant user impact and to predict short-time future expected behavior; and
- Last, but not least, running real-time RCA at the edge (on-device/edge computing) in TEMS drive testing tools (e.g. TEMS Investigation, TEMS Pocket).
One of the first prototypes has been developed for a connected car application, and it provides coverage and throughput (QoS/QoE) prediction with on-device ML solutions. As we can see in Figure 2, while driving along a route (counterclockwise), the driver gets real-time information on the network coverage availability (quality and technology), as well as QoS/QoE performance, that can be expected within the next few seconds, depending on the vehicle’s speed.
While a full solution for ML-based RCA is still undergoing research, TEMS already offers automated post-processing enabled by real-time data streaming, rule-based RCA and automated advanced reporting in TEMS Director for various 5G NR uses cases (e.g. NSA NR deployment roll-out analytics, SA analytics), as well as advanced scripting analysis capabilities (e.g. coverage/connectivity/RAN performance), in the TEMS Discovery post-processing solution to be discussed in future blogs.
3D testing
TEMS already has a drone-based solution for LTE and is currently working to evolve it for 5G NR with clients on the drone to be used for coverage and performance (throughput, latency) analysis for 3D outdoor as well as unreachable indoor locations, mainly for IoT use cases. Preliminary tests for data collection, analysis and 3D visualization have been performed on a 5G SA testbed network in Sweden.
Drive test set-up configurations within the context of 5G NR
Generally, the drive test set-up for 5G NR is similar to the one of LTE-A. Based on ETSI TR 103.581 (revised version, July 2017), devices have to be positioned in the car and working using their own internal antennas rather than roof top-mounted ones in order to accurately emulate the MIMO environment of a real-life scenario.
In addition, the larger 5G NR variety of spectrums and bandwidths configurations (e.g. with carrier aggregation, dynamic spectrum sharing) is expected to generate more interference artifacts which are likely to occur between two devices within too close a vicinity, and consequently can result in misleading testing measurements (KPIs). Therefore, 5G NR testing set-ups require more elaborate and thorough analyses of device positioning than for previous technologies when more than one device is used. TEMS test set-up configurations are carefully designed to cope with these kinds of artifact.
Last, but not least, unlike LTE test scenarios, 5G devices working in NSA configuration are likely to exhibit overheating, even if 5G NR is deployed in a sub-6GHz spectrum. This is expected due to the simultaneous transmission/reception in different bandwidths and spectrums. Therefore, when considering the position of a device, you should also take into account the overheating effect. Current investigations related to this topic are ongoing in various standardization organizations.
Flexible support to cope with a large variety of 5G devices and the testing of them
As a significantly growing number emerges, 5G devices all require careful testing for their behavior and interaction within the context of the real-life beam-centric 5G NR network, despite being lab-validated. To optimize and enhance for flexible support of 5G device variety, TEMS has two approaches in place: (1) TEMS devices with extensive ICD decoding; and (2) connected devices with limited decoding depending on specific testing needs and logging availability. TEMS also supports a third solution: the manually configurable device, for cases in which the device is unknown. TEMS also supports a set of 5G scanners, for initial spectrum clearing and other coverage and interference test use cases.
TEMS probes: in conclusion
TEMS probes’ methodology is a fast-paced discipline, which evolves with the emergence of 5G NR technology in a continually increasing number of network deployments.
In future blog posts we will start to share some of the sought-after test uses cases we learnt about while working with operators during their 5G NR deployments. Next up: testing mMIMO/3D beamforming techniques.
This blog post is based on a section of a detailed white paper, which you can download here.
