The crux of any release or sub-release is the question: Have all the necessary and planned test cases been executed? Coverages help answer this question. They instantly indicate the level of test case coverage for specific requirements, parameter ranges, or variants. A quick, trained look is often all that is needed to correctly evaluate the test results in the graphs and tables.

Coverages are defined using coverage filters. These filters define the boundary conditions for the test scope of a requirement, for example, the vehicle variant or engine class which is to be tested. Only the test executions that meet the defined filter conditions are sufficient, are monitored and included in the analysis.

Coverages can be created individually for each requirement. For this purpose, prefabricated examples are already available that can be modified for the respective filter definitions and make it easy to start. In addition, the underlying XML structure makes it easy to generate completely custom scripts for coverage filters. Filters can be imported to TEST-GUIDE from files and ALM systems (e.g., Polarion, HP-ALM, Octane), as well as automated via the REST API, e.g., for Continuous Integration solutions.

Essentially, it's about distributing a test job to the right test bench. Sounds simple at first. However, if thousands of test jobs need to be distributed simultaneously, automatically and continuously to scalable test resources, then full power is required. It is good that TEST-GUIDE is following the right strategy here.

TEST-GUIDE automates the execution of tests so that they are executed not only on suitably configured test benches, but also in the order of their prioritization. Added to this is the optimal use of the available test resources so that idling does not occur.

The ResourceAdapter is responsible for the latter. A software that is configured on the test benches and continuously sends information about vital data, configurations and test case execu tions to the central TEST-GUIDE server. This enables the test execution to decide in real time the test bench on which the next test job can be executed. In addition, the ResourceAdapter ensures that the test bench is prepared for the test. It checks out repositories, copies data back and forth, and clarifies other additional dependencies – all based on the description in the playbook.

Playbooks can be created in ECU-TEST and uploaded directly to TEST-GUIDE via the integrated interface. Direct creation in TEST-GUIDE is also possible. The workspace to be used – including test cases, parameterization of global constants, test bench configuration, and the cleanup after execution – is defined in it. Playbooks thus make the process of automated test job distribution to physical and virtual test benches even more convenient. That is, already when creating the test-suite. Continuous Testing? Done.

Especially with many test executions, it is extremely important to keep an eye on the test resources. The time on the test bench is too valuable to waste it without being used. Availability, utilization, disk storage capacity, configurations, ongoing test executions – all need to be maximized.
That's why we have deliberately added a little extra. Or more precisely, a small add-on: the ResourceAdapter. It is an agent that allows such a comprehensive monitoring of test resources. Once installed, it continuously sends all information from the test bench PCs to the central TEST-GUIDE server via the monitoring API. There, this data is combined and can be accessed in real time – in different view options, depending on your interests.

With these comprehensive details about the connected test resources, it is easy to determine which test bench is ready for the next test execution, for example in a Continous Integration environment. Or even the performance and duration during the last executions.

Normally, it is extremely time-consuming to manage, view and analyze test reports, i.e., the results of innumerable test executions. Not with TEST-GUIDE. The tool centralizes the results in a database and helps to restrict the relevant test reports for a release using many different filter and analysis options.

The test reports are automatically imported from ECU-TEST. However, it is also easy to import for user-specific test tools thanks to the standardized ASAM-ATX interface. As soon as the test reports are available in TEST-GUIDE, the test runs can be analyzed in a targeted manner. In addition to test schedules and test statistics, comprehensive time and event-based filter options are available for this purpose.

Compare test runs and regression tests, trace test steps, identify trends, assess defects or create overview diagrams. The result filter is the first choice here. It is the multi-functional tool when details matter. Set filter properties to design custom visual dashboards that display test results specifically as needed. The team also benefits from this. Because each filter can be subscribed, exported or shared via email.

The comprehensive review function provides a lot more extra. It can be used to record notes for failed test runs, perform re-evaluations or even create defect tickets. The latter is easily possible via the integrated link to ticket systems, such as Jira, Redmine, HP ALM.net or Octane. Various export options (PDF, Excel, test management tools, ATX documents and full backups) are also available to share or save any form of result reports. For continuous flow of information at any time and from anywhere – both from the workplace and on the move via mobile devices.

The release encompasses all test procedures just like a set of brackets. Connects the beginning and the end. Compares actual status with target status. Helps to make decisions. Even the decision of whether a release can be closed. This happens when the initially defined test scope has been successfully implemented. The workflow for a release is almost always the same. Create > Monitor > Detect irregularities > Upgrade > Add if necessary > Check > Close. The difference is in the creation. This is because a release can be created in advance or afterwards.

If the release is based on tests that have already been executed, then relevant test reports are filtered via the results filter and assigned to a release. However, a release is often created before executing the first test case, i.e., during the planning phase. Then a defined coverage filter with a certain scope of requirements becomes the basis. All test reports that meet these requirements are automatically assigned to the corresponding release. For as long as it takes until the release is closed.

Once a release is ready, success is measured in terms of tests passed and advanced coverage. The release overview keeps you optimally informed about the status – both graphically and in tabular form. In addition, it is easy to navigate from here to any release-relevant section of TEST-GUIDE with a single click. Practical! Especially when there are several releases to organize at the same time. If the release was closed successfully, the results can be exported in different formats (ALM systems, e.g., Polarion or Octane, Excel, PDF).