Real compliance of mobile devices with technical standards is the key to prevent interoperability issues. As this is so important, the industry has defined very robust methods to ensure this by using validated test tools such as those provided by COMPRION.

We’d like to give you some behind-the-scenes insights into what it takes from the definition of the technical specification to the delivery of an approved test tool. As an example, let’s look at the eSIM function of mobile devices as specified by the GSMA:

1. Defining the technical specification
   The GSMA (the mobile industry association) facilitates cooperation between all interested industry players in defining a technical specification. The result is an agreed stable document, in this case: GSMA SGP.22. COMPRION’s GSMA delegate participates in this process to ensure we have a clear understanding of all details at the earliest possible stage.
    
2. Defining the test specification
   The next step is to derive a test specification in relation to the technical specification, i.e. creating a list of test cases (typically 100s) with defined pre- and post-conditions which allow comprehensive conformance testing of an eSIM device. Besides already supporting the engineering process of the device manufacturers, it will also become the basis for formal compliance testing. COMPRION is usually significantly involved in the processes for defining a test specification and often even leads them, here: GSMA SGP.23.
   
   COMPRION’s product manager for the eSIM/RSP products is simultaneously planning to deliver a corresponding automated test solution as part of our product roadmap.
    
3. Implementing the test tool
   As soon as the test specification is sufficiently stable, COMPRION’s software development team starts to implement a matching test bench on an existing test management platform (in this case, Connectivity Test Center). This platform takes care of, for example, automation, reproducibility, efficiency, analysis and identification of the root cause of a problem, documentation, and sharing of test results.
    
4. Adding a new test scope to a conformance test framework
   The industry initiative that defines and governs the conformance test framework for mobile device testing is the Global Certification Forum.  It represents parties from different industries, such as mobile device manufacturers, mobile network operators, test tool providers, and test laboratories.
   
   Members who want to ensure that eSIM device features are part of a solid conformance testing process will launch a new work item. Here, the participants agree on a set of test cases from the test specification that will be added to GCF’s conformance test framework for mobile devices. COMPRION’s representative participates in GCF meetings to make sure we clearly understand the work item requirements during its introduction and description process, in this case: GCF WI-274 and GCF WI-276.
    
5. Validating the test solution
   As soon as the test solution has been implemented, COMPRION will seek its validation. Without meeting the Test Platform Approval Criteria (TPAC), the test tool cannot be used for official GCF conformance testing.
   
   This is done by a “validation organization”, which is a test laboratory approved by the GCF for this task. During the validation process, other GCF member companies have the opportunity to get validation logs and subsequently challenge them. If the conflict between challenging party and challenged party cannot be resolved, the GCF and the conformance agreement group are consulted. Only afterwards the GCF will declare the test platform and the test cases as validated.
    
6. Deployment in test laboratories
   GCF Recognized Test Organizations (RTO) can then get the approved COMPRION test solution and use it for conformance testing of mobile device with regard to the eSIM aspects.

The outlined process guarantees solid conformance testing that leads to interoperable devices. However, as you can see, this is a lengthy and complex process that requires considerable effort. The duration of such a process after a technical specification has been defined (i.e. steps 2–6) may take a year (order of magnitude).

Would you like more information? Then contact Marcus Dormanns at MDormanns(at)comprion.com.