Prior to my experience in an avionics environment, I had never heard of MC/DC, now I awake in the night shaking.
Modified-Condition / Decision-Coverage is one of the verification analyses that must be performed upon all Level-A code in a DO-178B environment.
Level-A is defined by DO-178B as “software whose anomalous behavior…would cause or contribute to a failure of system function resulting in a catastrophic failure condition for the aircraft”; in other words, it’s software that could kill. Lesser levels can make things difficult for the crew, but Level-A, either by direct control or by providing faulty data to the crew or other computers has the potential to be deadly. Levels are designated A – E, where each level has progressively fewer requirements. Level-E, having no safety impact, is fully unregulated.
Because of the expense and difficultly of performing MC/DC analysis, DO-178B mandates it only for Level-A software. It may be applied to other levels at the discretion of the developing organization.
The goal of software verification is to ensure that the software end product functions in accordance with it’s requirements. DO-178B further aims to ensure that safety-critical software doesn’t deviate from requirements under any condition. In order to do this, they require various forms of coverage analysis, which is intended to verify the effectiveness of the verification activity. All tests are expected to be requirements-based; that is created according to the requirements, without knowledge of the implementation. After running all tests on a module, the results are analyzed to determine what executable structures have been exercised by the tests. If the software functioned properly and full coverage was attained, then you are pretty much done; but for a larger project, it’s not likely to be that simple.
Many things can affect these analyses, and one of them is code written for re-use or multiple configurations. DO-178B allows for the existence of deactivated code, but there is disagreement even among DERs I’ve spoken with as to the difference between dead and deactivated code. A library function that is never called may be there by design, but if there is a requirement for it some argue that it must be verified even if it can never be activated, and verification and coverage cannot be achieved in a system context. In my opinion, the requirement to prove that it cannot be activated should supplant the requirement to verify its functionality, but that opinion is by no means the only one.
Another practice that affects coverage is defensive programming. Although an acknowledged best practice in most industries, it becomes controversial in the aviation world. DO-178B says little about defensive programming except that it “may be considered to improve robustness”; but it produces holes in statement and condition coverage that the standard says must be met. Writing a simple else statement to handle a situation that turns out to be prevented elsewhere creates a hole. Remove the else statement to satisfy coverage, and someone else makes a change to the preventative because the requirement failed to mention it; and there is no coverage hole, but no robustness handling a condition that may later cause system instability. This sort of situation happens more than you might imagine. Some organizations with a more liberal stance on deactivated code simply justify the coverage holes caused by defensive programming as design intent; proven by the coverage hole to be deactivated.
Structural coverage analyses are mandated only for software at Level-A and Level-B; and MC/DC is the only one of these that is excluded from Level-B. Below we will give some definitions, quotes reflect actual wording from DO-178B. Note that each analysis is entailed by those that follow it.
Data Coupling and Control Coverage: This is not defined in the DO-178B glossary. Data Coupling refers to data transfers between components, and control coupling, similarly refers to transfers of control, as in a call/return or a co-routine yield. As such, I would interpret that this coverage analysis should ensure that each statement affecting a transfer of control or data is executed, and that each point of entry or exit have been exercised.
Statement Coverage: Every statement has been covered.
Decision Coverage: Each branch (decision) within the program has been covered for every possible outcome, and every entry (call) and exit (return) have been taken.
MC/DC: “Every point of entry and exit in the program has been invoked at least once, every condition in a decision in the program has taken all possible outcomes at least once, every decision in the program has taken all possible outcomes at least once, and each condition in a decision has been shown to independently affect that decisions outcome. A condition is shown to independently affect a decision’s outcome by varying just that condition while holding fixed all other possible conditions.”
Now, if that last point didn’t sound too bad to you, consider that NASA thought that it warranted this 96 page tutorial on the topic.
The purpose of MC/DC is to verify branch coverage for the branches you can’t see; those that occur at the assembly language level. Each term of the boolean expression generates a branch in assembly language. If any part of the full expression is not independent of all others then some of those assembly language branches will not be taken.
There are 3 different flavors of MC/DC analysis: they are, from weakest to strongest, Masking, Unique Cause, and Unique Cause with Masking. Interestingly, a study done on behalf of the FAA to investigate the differences determined that the weakest, and easiest to conduct form, Masking MC/DC, gave results that were statistically equivalent to the strongest form. This caused the Certification Authority Software Team (CAST) to issue a position paper (CAST-6) advocating the use of Masking MC/DC. Prior to this, many were of the opinion that it did not meet the requirements of DO-178B.
The upshot is this: If you need to worry about MC/DC, be very careful of the structure of complex conditionals. Essentially the best bet is to ensure that every expression is reduced, via boolean algebra, to its simplest form, and ensure that each term can be independently varied. That sounds easy, but sometimes the simple things are forgotten.
MC/DC may be an after-thought in some organizations, something that verification will check with automated tools when development is done. This mindset results in far more errors being induced, as the developers are not aware of the issue while the development is still in its early stages. MC/DC should always be checked by developers during code reviews, and verified by the verification team.
By the way, if your plans include developing software under the upcoming DO-178C, you should know that there are those who believe that Modified Condition /Decision Coverage is lacking in some respects. Reinforced Condition / Decision Coverage has been proposed as a stronger replacement.
Max is a father, a husband, and a man of many interests. He is also a consulting software architect with over 3 decades experience in the design and implementation of complex software. View his Linked-In profile at http://www.linkedin.com/pro/swarchitect |
Comments