The survey questionnaire that our team has designed aims to gather insights into the clinical use of a therapeutic intervention in oncology / palliative care settings. The questionnaire contains both open and close ended questions. Our aim is obviously not to measure outcome measures for clinical assessment / diagnosis or screening purpose. I will though be pilot testing the questionnaire on a small sample, to identify and to eliminate potential problems.
I am well aware of validity and reliability testing measures, my quandary is - do I need to do it?
Thank you in advance for your kind help.
PS the study's aim was to gather descriptives (cross-sectional snapshot) and NOT draw inference or generalise findings
Hi Gourav a previously tested study is a good place to start to establish that you have accounted for issues of validity and reliability in your research. However, it is always good to test for these none the less. Especially where your context differs from that of authors of prior work. E.g. If your study will be done in an emerging economy and the prior work was done in a developed economy - will the instrument be relevant. Is the language still applicable? Are the scales adequate? The best way to answer these questions is by doing a pilot survey.
The most commonly reported test for reliability is Cronbach's alpha, which you could apply in any situation where you have a multiple item scale. it is easily available through most statistical analysis software, and your reviewers will almost certain appreciate having that information.
A new designed questionnaire as a research tool need to be validated and tested for reliability, Therefore, it is strongly recommended to conduct a pilot study to ensure both validity and reliability.
Yes, the pilot study validates and tests reliability of the questionnaire.
Simple answer - Yes
Any statistics you generate or inferences you make will have little merit if the data that generated them is not reliable. Rule of thumb is cronbach's alpha should be >0.7 for scale data to be considered reliable
pilot testing of a tool is important to test the functioning of items in questionnaire. it will be better to evaluate the questionnaire through experts' opinion method before pilot testing of your tool. so, according to experts' opinion you can improve your items in questionnaire and ensure the validity of items in the tool
Yes. for newly designed questionnaire, you need to validate to ensure that you are measuring what you intend to measure and it is reliable. You may start with content validity by asking opinion from experts in oncology before piloting the questionnaire.
Yes, as doctoral students we conferred with one another on our pilot studies prior to implementation. After the pilot study, the questionnaire was again revised according to feedback.
Thank you all for your kind assistance; your advises have been helpful. I am going to validate the questionnaire using face + content (CVI) + cognitive validity (think aloud). Reliability tests like Cronbach's alpha I reckon will not be of much use because our questionnaire has only 1 multiple scale item.
Cheers
Yes, it is good practice to test the pilot study / initial survey questionnaire for reliability and validity tests. The rationale can be:
1) the questionnaire might be newly developed by the researcher to represent a construct in which reliability & validity need to be ascertained before actual data collection,
2) may be the questionnaire is reliable & valid in other countries / industry / field of study but since it is being used in local country / other industry / other field of study - it is prudent to recheck its reliability & validity before moving forward,
3) may be the current research is unique / first of its kind in which there is no previous re-useable questionnaire whereby the researcher needs to adapt the questionnaire from other field of study.
4) may be a new construct is being developed by the researcher - i.e. amending & combining 2 or more different constructs together,
5) may be due to unique research requirement - a construct is required to decouple & amend into several sub-constructs / smaller questionnaires (factor analysis can be used) in which each smaller questionnaire needs to be tested for their reliability & validity.
- Badges
- Science topic
- Similar topics
- Statistics
- Survey