I recently concluded a Modified Delphi Study in which I developed a research instrument. The instrument had a high consensus rate as expert participants reached agreement on the issues explored. I however want to use the instrument to further explore stakeholder's perspectives regarding an issue and I would like to know if it is necessary to pilot test the instrument or rather pretest it? I am stuck between making the decision to either pilot test or pretest. My perspective is that I may not have to pilot test since the instrument was developed from a Delphi study with a high consensus rate, but then again does a high consensus establish reliability. Or do I simply pretest it to establish readability and understand-ability from my sample? Please I need your perspectives with corresponding justifications to support it. Thank you
In my opinion high consensus among experts is an acceptable level of reliability. However, being a Ph.D study additional test will further consolidate the reliability. Pretest will be appropriate for the purpose of getting exact quality result.
Dear Caleb Chinedu,
It seems you have already done some kind of pre testing/ pilot testing. However, I would always recommend to pre test the instrument/s before going in to detail data collection as always it has advantages than disadvantages.
There is a difference between high consensus about the topics to be studied versus similar agreement on the wording of the questions that you will use to study those topics. If your experts did not evaluate the actual questions that you will use, then I would indeed recommend some kind of pilot or pretesting.
Open option would be to do "cognitive interviews" with a relatively small number of people.
- Badges
- Science topic
- Similar topics
- Methodology
- Research Methodology