Multimedia testing

Infotainment

Today's vehicles contain many new features such as real-time traffic information, web radio or cloud-based personalization. But how does the specialist department safeguard these new software components in development? Software has been tested in the vehicle for decades. For a long time, the focus was only on safeguarding the drive components, where minor errors in software functions have no bearing on the functionality of the overall system.

When testing the functions in the infotainment area, this is precisely the challenge. This is because the vehicle occupants notice even the smallest error in the system during operation. Whether it is a connection failure with a smartphone, a crash of the music app or a button that is not reacting. The functional diversity of different software manufacturers, countless operating sequences and time-dependent behavior make manual testing almost impossible here. However, until recently, this was exactly the approach that was used. Instead of automating, more test benches were purchased and even more testers were hired. The massive problems in safeguarding led to the big bang and thus to a rethink in the sector. There is no way around the automation of tests and the generation and parameterization of test cases.

Show content

Test systems of today for the infotainment of tomorrow

With the enormous increase in the complexity of integrated functions in the vehicle, manual function tests for each new software version are no longer practical. The primary goal of the project is therefore to provide a self-sufficient test system that executes fully automated tests of new software components on different test stations via suitable interfaces and simultaneously provides several possibilities for evaluation – including real vehicle data. In this way, an infotainment developer receives quick feedback on his new developments. The entire test process, from planning to reporting and error tracking, is saved in a central database in a permanently traceable manner. ...

But how does that work exactly? Further and new developments of apps in the infotainment area must also be compatible with older vehicles, since a vehicle has a significantly longer product life cycle than an app version. Due to the large number of embedded systems on which the apps have to work, new functionalities have to be tested extensively both on test benches and in real vehicles in order to be able to offer customers an error-free infotainment system.

If a new app is developed or an existing one is extended, the developer defines both a test case for the new function and boundary conditions that are needed to execute the test. The test system immediately recognizes the new test case, which is stored in TEST-GUIDE and integrated into the execution planning. When the changes to the new app version are released, the test system starts automatically and the test procedure begins.

With the interfaces developed by us and with the help of the test requirements of the app developer, the required real and virtual environments are now independently selected and all test cases stored in connection with this app are executed with ECU-TEST. The challenge for us is to run the tests in parallel on as many (50+) test benches as possible with different environments.

The complexity of the necessary tests is based on the permanently changing test objects – the apps, the backend and the user's operating modes. Therefore, the following tests are performed as standard:

Operation test: How do users operate the infotainment system?
Backend test: Can digital infrastructures/central servers be permanently accessed?
Load test: How do the backend and app behave during continuous or unsystematic use?
Regression test: Do the new app features have cross effects that cause errors in existing apps?

The test results generated in this way are saved directly in TEST-GUIDE and are permanently available for evaluations and new test planning. A test report compares all tests performed with the requirements. The new function is released only when it passes both the existing test cases and the new test case.

We can fully integrate and evaluate the results of tests that cannot be automated (e.g., tests at the customer's site or in test vehicles) into the created infrastructure.

But we're still not where we want to be. Our project is not yet completed. But the interim result shows that a lot has already been achieved.

Test results from customer vehicles, test stations and test vehicles are combined in one system.
All test scopes for test stations are fully automated.
A continuous test chain for apps and backend is established.

We are working at full speed to provide infotainment developers with a testing tool that gives them rapid feedback on the software components they have implemented.

	App	Application on the infotainment system in the vehicle (e.g., weather forecast, music streaming) or on a smartphone (e.g., digital car key)
	App version	Identification of the development stage of an app in the form of a version number according to the type MAJOR.MINOR.PATCH (e.g., Browser 5.0.4)
	(Mobile) Backend	Cloud computing architecture that gives mobile applications access to servers, storage systems, databases and other resources which they need to be operational
	Operating modes of the user:	Nowadays in the vehicle – especially touch operation of a display, voice input, input via smartphone

6 members in the team
Distribution over 2 locations
FunFact: 500 song downloads in the ConnectedMusic test

1 Data engineer
1 Software architect
1 Automotive software tester
2 Software developers
1 Computer science student

Show content

From zero to Continuous Testing

Infotainment systems are already an elementary component of a vehicle today – and create an enormous need for testing in all possible directions. Unfortunately, hundreds of specified tests and test stands cannot be automated at once. A transition strategy with semi-automated test variants is required, which is then gradually supplemented by further automatisms. The objective is clearly Continuous Testing, because this is the only way to detect non-deterministic behavior and sporadically occurring error effects. ...

In the past, the control units of different domains hardly needed to communicate with each other. This has changed a lot in the era of infotainment systems. The task now is to integrate ECUs from different manufacturers with different software into the overall system in such a way that communication via CAN and Ethernet runs smoothly. If, for example, messages are not transmitted to an infotainment system fast enough, this leads to an overlapping of image content. Initial manual tests thus consist of checking the general stability of the infotainment system: Do all functions in the infotainment screen menu load when the engine is started? How long does it take until the system is ready for operation? These tests have to be repeated countless times and are still an integral part of test procedures. In the meantime, these functional tests, like many others, can be automated. It's a good thing that automation is exactly where our passion lies.

We are of the opinion that problems must first be experienced first-hand before you can work constructively on a solution. That's why we use ECU-TEST in the daily test process and also continuously perform automated functional tests, such as smartphone coupling with the infotainment system, directly using our Jenkins plug-in for ECU-TEST. The trace analysis (TRACE-CHECK) integrated in ECU-TEST also makes it easy to check whether all intermediate steps on the communication path have been reached and whether the test run was error-free.

If we encounter errors in the test process or if new components are needed for test case automation, the close cooperation with our product development department is of great advantage. Because, on the basis of our work in customer projects and the use of our own tools, we can influence the further development of our products. In this way, even tricky customer wishes can be realized and we contribute together to new, better solutions.

Our goal in the customer project is to provide an infrastructure (ECU-TEST, Jenkins, TEST-GUIDE) that allows testers without expert knowledge to automate and continuously execute test cases.

CAN/CAN-FD
Automotive Ethernet/SomeIP/VIWI
Video (Image and text recognition)
Audio

Interdisciplinary team of 20 engineers, computer specialists and scientists (d/f/m)
Divided into four expert teams for:
-Automated request-based tests
-Automated stability tests
-Automated trace analyses
-Cloud Infrastructure