The investigators here are searching for a way of saying that the test matched its requirements, met the expectations of the standards that are applicable, and that the test was high quality. They drew the term from HW tests. All obviously overlapping with sense of test validation, test coverages (of many sorts), and other test effectiveness measures. Have any of you developed a metric on the fit of a test against requirements, standards, etc.? Any references very welcomed.
Kristie:
Does this question apply only to a software system?
Or does it apply to a "software and hardware" combined system?
Or does it apply to a 'software, hardware and wetware" system?
Take care and have fun,
Joe
Kristie:
Lets assume that the metric covers the total range of system components.
Further lets assume the requirements are not static bu dynamic, like a dynamic threat environment.
I developed the "Secure Adaptive Response Potential (SARP)," a system security metric to focus on all aspects of security requirements.
You may be able to adapt the SARP approach to other areas.
A link to the SARP paper is:
http://www.systemsconcept.org/static_files/2008/INCOSE08_SARP.pdf
Take care and have fun,
Joe
I see that you work in aerospace. Tests are thorough if the tester is willing to be a passenger on the test flight!
The "lines of code covered" metric is poor, because the test suite may execute a line without validating the logic. Mutation coverage is a much better metric - if you have a code mutator and your test suite detects error in 90% of the mutant code bases, then you have some basis for claiming that your tests have also discovered 90% of the real errors.
- Badges
- Science topic
- Similar topics
- Human-Computer Interaction
- Collaboration