It is customary to do a pilot test or trial of a new program or product before going live with it. There is the understanding that this pilot test would help to identify logistical and effectiveness issues for what is working or not, so improvements can be made. But, is there a specific theory or methodology for conducting a pilot test?
Pilot study is a function of focus of your research work you can apply a prototype framework depending on the area of focus
Typically, a pilot study should adhere to the methodology planned for the main study. The idea is to dry run or test the feasibility of conducting the study using planned instruments and procedures (i.e., method). For example, one criteria may be to assess whether the survey instrument operates in practice as planned. Other criteria may be to test whether a small sample provides data suitable (or not) for conducting appropriate statistical tests. Are the questions delivering appropriate data? etc.
I cannot emphasize enough the importance of have a set of requirements (put under configuration management) that are SPECIFIC AND TESTABLE. Then you can use the technique of a traceability matrix to measure consistency through all the stages of the project development. Way too often the "customer" changes requirements at some stage of development, and this causes big challenges for redesign, recoding, and reintegration of the modules, as well as the documentation. It changes the test planning as well. At the Johns Hopkins Applied Physics Laboratory, I developed a metric for measuring project completion status that took away project completion "points" if a requirement was changed. We then saw critical projects supporting the Navy's ballistic submarine evaluation system actually going BACKWARDS using the "Scott SW Status" metric. Needless to say, APL killed my reporting and refused to tell the Navy the real status. After a couple more years there, I resigned and went into consulting. I have not published my metric formula, but probably I should.
Leighton's right. If you cannot write a set of requirements specs you're cutting water.
But if you are genuinely agile you would already be witoing tests as you go and executing them every time some funcitonality is added - and you would never have asked this question. So you're not. So when you've written the requirements you willl know enough to write pilot or any other tests.
To be less terse. There are two kinds of tests: those which show it works and those which attempt to show it doesn't. There is, if not an infinite number of the latter, at least a very large number. The first are (hopefully) trivial, the latter require much invention. If you have a list of requirements then you can assess the degree of coverage you need. A rough measure of coverage is: one positive test of every feature and two negative ones. What's a feature? You decide.
A closely related and somewhat overlapping term and process is prototyping (and/or "rapid prototyping") The key value of getting some semblance of the system or subsystem working early is in getting user interfaces and system functionality in front of the user as soon as possible. This helps in refining and nailing down those pesky system requirements that we talked about before. Also, the software architecture (as can be documented in my "Leighton Diagrams") can begin to be formalized. Finally, in today's world, the system process should emphasis modular design and provide additional time for performance and security testing.
Yup, been there done that. It's a great way of getting user involvement (since users tend not to read much). But having established a systems demonstrating features you still have to build the real thing - and then test that.
PIlot test is the physibility of the system developed whether frame work prepared testing instruments and soft ware develped give results.
Yes Matthew, I think this is a process of evolutiuonary understanding. Also see
Relative to survey methodology, a preponderance of theory and methodology can be employed to pilot new or modified instruments (e.g., Folwer, 2009; Swanson & Holton, 2005; Creswell, 2009). Even in qualitative research, piloting your interview questions and procedures should be based on sound methodology (Swanson & Holton; Creswell).
Fowler, F. J. (2009). Survey research methods (Vol. 4th). Thousand Oaks, Calif.: Sage.
Swanson, R. A., & Holton, E. F. (2005). Research in organizations: Foundations and methods of inquiry. San Francisco, Calif.: Berrett-Koehler.
Creswell, J. W. (2009). Research design: qualitative, quantitative, and mixed methods approaches (3rd ed). Thousand Oaks, Calif.: Sage.
- Badges
- Science method