Functional testing of infotainment systems is particularly challenging. On the one hand, the range of functions is constantly increasing. On the other hand, the vehicle occupants notice even the smallest error in the system during operation. Whether it is a connection failure with a smartphone, a crash of the music app or a button that is not reacting.
We have initiated several engineering projects to automate infotainment testing in order to manage the testing effort.
Infotainment testing
Voice control, smartphone coupling, web radio, favorite songs, navigation system – and all of this preferably in parallel? Sure!
Test systems
Test systems of today for the infotainment of tomorrow
Further and new developments of apps in the infotainment area must also be compatible with older vehicles, since a vehicle has a significantly longer product life cycle than an app version. Due to the large number of embedded systems on which the apps have to work, new functionalities have to be tested extensively both on test benches and in real vehicles in order to be able to offer customers an error-free infotainment system.
What is the project's objective?
The primary goal of the project is therefore to provide a self-sufficient test system that executes fully automated tests of new software components on different test stations via suitable interfaces and simultaneously provides several possibilities for evaluation – including real vehicle data. In this way, an infotainment developer receives quick feedback on his new developments.
The entire test process, from planning to reporting and error tracking, is saved in test.guide in a permanently traceable manner. A test report compares all tests performed with the requirements. The new function is released only when it passes both the existing test cases and the new test case.
The challenge for us is
- to run the tests in parallel on as many (50+) test benches as possible with different environments
- to fully automate all test scopes for test benches
- to establish a continuous test chain for apps and backend
- to combine test results from customer vehicles, test benches and test vehicles in one system
How do you proceed when testing?
If a new app is developed or an existing one is extended, the developer defines both a test case for the new function and boundary conditions that are needed to execute the test. The test system immediately recognizes the new test case, which is stored in test.guide and integrated into the execution planning. When the changes to the new app version are released, the test system starts automatically and the test procedure begins.
With the interfaces developed by us and with the help of the test requirements of the app developer, the required real and virtual environments are now independently selected and all test cases stored in connection with this app are executed with ecu.test.
The complexity of the necessary tests is based on the permanently changing test objects – the apps, the backend and the user's operating modes. Therefore, the following tests are performed as standard:
- Operation test: how do users operate the infotainment system?
- Backend test: can digital infrastructures/central servers be permanently accessed?
- Load test: how do the backend and app behave during continuous or unsystematic use?
- Regression test: do the new app features have cross effects that cause errors in existing apps?
Tools and technologies
- test.guide
- ecu.test
- trace.check
- Jira
- Confluence
- Python
- Groovy/Java
Key figures
- 6 members in the team
- Distribution over 2 locations
- FunFact: 500 song downloads in the ConnectedMusic test
Team
- 1 Data Engineer
- 1 Software architect
- 1 Automotive software tester
- 3 Software developer
- 2 Computer science students
CI/CT
From zero to Continuous Testing
In the past, the control units of different domains hardly needed to communicate with each other. This has changed a lot in the era of infotainment systems. The task now is to integrate ECUs from different manufacturers with different software into the overall system in such a way that communication via CAN and Ethernet runs smoothly. This requires an enormous need for testing in all possible directions. Unfortunately, hundreds of specified tests and test stands cannot be automated at once.
What is the project's objective?
Our goal in the customer project is to provide an infrastructure (ecu.test, Jenkins, test.guide) that allows testers without expert knowledge to automate and continuously execute test cases.
The focus is clearly Continuous Testing, because this is the only way to detect non-deterministic behavior and sporadically occurring error effects.
Strategically, we rely on semi-automated test variants, which is then gradually supplemented by further automatisms.
We are of the opinion that problems must first be experienced first-hand before you can work constructively on a solution. That's why we use ecu.test in the daily test process and also continuously perform automated functional tests, such as smartphone coupling with the infotainment system, directly using our Jenkins plug-in for ecu.test.
The trace analysis (trace.check) integrated in ecu.test also makes it easy to check whether all intermediate steps on the communication path have been reached and whether the test run was error-free.
How do you proceed when testing?
If, for example, messages are not transmitted to an infotainment system fast enough, this leads to an overlapping of image content. Initial manual tests thus consist of checking the general stability of the infotainment system:
- Do all functions in the infotainment screen menu load when the engine is started?
- How long does it take until the system is ready for operation?
If we encounter errors in the test process or if new components are needed for test case automation, the close cooperation with our product development department is of great advantage. Because, on the basis of our work in customer projects and the use of our own tools, we can influence the further development of our products.
Tools
- test.guide
- ecu.test
- trace.guide
- OpenShift
- Docker
- Jenkins
- Python
- Sphinx
- Git
Measured data analysis
- CAN/CAN-FD
- Automotive Ethernet/SomeIP/VIWI
- Video (Image and text recognition)
- Audio
Team
- 20 engineers, computer specialists and scientists for:
- Automated request-based tests
- Automated stability tests
- Automated trace analyses
- Cloud Infrastructure
Video favored?
Infotainment testing with ecu.test
We get a lot of questions about infotainment, for example:
Lukas and André show you how infotainment testing with ecu.test generally works in the video on the right.
- How do you analyze electronic dashboards?
- How do you analyze multimedia functions?
- How do I start analyzing the HMI?
- How can I get access to the system to be tested?
- ...
Lukas and André show you how infotainment testing with ecu.test generally works in the video on the right.