Sunday marks the end of the 2022 NFL season with the stupendous spectacle that is the Super Bowl. So, it should be no surprise that (American) football jargon was in the ConsuNova air this week. The “Power sweep,” “nickel package,” and “icing the kicker” are just a few of the phrases expressed on “any given Sunday,” though one stands out as an avionics and aerospace crossover: “the coverage sack.” For good measure, put simply, a “coverage sack” occurs when the defending team tackles the quarterback because no receiver was “open” to receive a pass. Though the analogy is a stretch, structural coverage analysis in DO-178C verification can be an equally veritable game-changer for avionics software.
There’s a lot to unpack in structural coverage analysis, but for this article we’ll stick to the key steps for DO-178C guideline acceptance, including data coupling and control coupling.
Firstly, at its core, structural coverage analysis is used to demonstrate that software has been thoroughly tested, so much so that no unintended functionality could possibly occur in the code. Data coupling and control coupling coverage analysis, demonstrates that the data coupling and control coupling interfaces in DO-178C software have been sufficiently exercised during testing. Both of these activities are crucial to a project’s verification and ultimate completion.
DO-178C requires evidence to demonstrate that the code has been covered by requirements-based testing. Either source code or object code can be tested, and each type of testing has its benefits and drawbacks, as described in Source code vs. object code structural coverage analysis. The purpose of coverage (often called structural coverage to be distinguished from requirements coverage) testing is to ensure that an application has no unintended functionality.
The expectation when applying for DO-178C compliance is that 100% of the code has been executed during testing or that, in cases where this is not possible (e.g. for defensive code that cannot be executed under normal conditions), the reason this code is untested is justified.
As you’ll need to document the method by which you plan to verify the structural coverage of your software testing in your DO-178C planning documents, you should evaluate methods early during your DO-178C planning. If you choose to use any structural coverage analysis tools, you should also evaluate these early.
Structural coverage can be analyzed at different granularities. The broadest granularity of analysis considers the proportion of functions in the code that have been called, while the narrowest granularity considers the proportion of conditions in the code that have been shown to independently affect the outcome in the decisions in which they occur.
Structural coverage testing can be done manually, but this is a laborious process, so automated structural coverage analysis tools are often used. Coverage can either be tested at the source code or object code level, and each type of testing has benefits and drawbacks. Structural coverage analysis in DO-178C The need to perform structural coverage analysis for DO-178C compliance is listed in 3 DO-178C objectives
Structural coverage can be collected from system tests, integration tests, and lower level (component) tests, so long as each type of testing represents requirements-based tests. Higher level testing (such as system testing) generally produces more structural coverage per test than lower-level testing (such as component testing) does. As such, it is common practice to generate as much coverage as possible through higher-level tests before running lower-level tests to cover sections of the code that haven’t already been covered.
Usually, the best approach to structural coverage analysis is to develop and run tests on-host during the early stages of the project then repeat them on a Model A/B system when the tests are complete and such a system is available. When collecting structural coverage results from system level tests, it may only be possible to do so in Model B or C
environments as system tests need all devices in the system to be available.
Just like properly executed coverage can win football games, for DO-178C compliance, properly executed coverage analysis ensures project success. To “go deep” on structural coverage analysis for DO-178C, check out our whitepapers.