What factors should we consider in the validation? What steps should we follow? Is there any established theory, model or guidelines for doing that?
That's a bit tricky. It depends on the context (what does the process address, which (kind of) company, what shall be improved, etc.).
IMO, a multi-method (mixed-method) approach as for instance described in
P. O’Leary and I. Richardson. Process reference model construction: implementing an evolutionary multi-method research approach. Software, IET, 6(5):423–430, 2012
is a good idea. Unfortunately, there is no "real" pattern to evaluate a new process. Hence, you often find (massive) case study research, e.g.,
A. Raninen, J. J. Ahonen, H.-M. Sihvonen, P. Savolainen, and S. Beecham. LAPPI: A light-weight technique to practical process modeling and improvement target identification. Journal of Software: Evolution and Process, 25(9):915–933, Sept. 2012
Also, experimentation can be done:
M. Kuhrmann, D. M. Fernández, and A. Knapp. Who Cares About Software Process Modelling? A First Investigation About the Perceived Value of Process Engineering and Process Consumption. In PROFES 2013, pages 138–152, 2013
Simulation can also be found. Anyway, there are several aspects to be considered in process validation, e.g., quality of the process (model), feasibility of the model, acceptance, user satisfaction - and much more. It really important to define the information you want to gather. Also important is a clear research design and a rigorous reporting (this is where many contributions lack the most).
Maybe, you also consider having a look at the ICSSP conference series (International Conference on Software and System Process: http://www.icsp-conferences.org/icssp2014).
You could use the ISO/IEC SQuaRE series to compare others development process using a practical scenario by establishing evaluation metrics according to quality in use and quality software product models.
There are some measures by which process effectiveness can be validated. One way is to see whether defects are contained (i.e. detected, fixed etc) in each phase (like requirements, design, coding , testing etc) so that defects are not carried over to next phase.
For more such measures, please see website : http://www.bth.se/com/mun.nsf/attachments/Metric%20examples_pdf/$file/Metric%20examples.pdf
Great answers! Just to point out that validation should be done based on several approaches, which SHOULD return approximately the same result.
- Badges
- Science topic
- Similar topics
- Engineering
- Modeling