Dear all,
I had prepared a questionnaire to assess the knowledge, attitude, and practices of HPV vaccine recommendation among physicians in south India. I had done content validity by using expert panel. Now i want to test reliability. I had used questions having multiple options, Likert scale responses, Binary answers, and description as answers. Now i want to test reliability of the questionnaire.
What type tools or statistics i have to be used to assess the reliability of questionnaire.
Hello Narayana,
The answer depends on what score(s) you intend to use, and how you intend to use them.
For example, if you believe that all the items in your questionnaire relate to one, central characteristic (this, by the way, is something you can empirically determine via factor analysis), and you intend to use total or averaged scores (across all items) as the value of interest, then either internal consistency reliability estimates (e.g., Cronbach's alpha; McDonald's omega) or item response theory models can help you determine the suitability of total scores and how well individual items contribute to that score.
On the other hand, if you were trying to measure whether attitudes about HPV vaccination are subject to change (via training, education, or improving medical personnel awareness), then you'd likely want to establish a measure of how stable the scores are over time (in the absence of intervention). This is sometimes referred to as test-retest reliability.
Generalizability theory offers a nice way to assess these and any relevant dimension you might envision towards determining how much/little score variation is dependable towards your universe of interest. Among people who use Rasch analysis (strongly related to one-parameter logistic IRT), Michael Linacre's FACETS program is the closest analog to a GT framework.
Here's a pretty straightforward starting reference that you might wish to consider as you continue your work: DeVellis, R. F. (2017). Scale development: Theory and applications (4th ed.). Thousand Oaks, CA: SAGE Publications.
Good luck with your work.
You may not be able to include all of these resources into a single scale score, due to the wide variety of ways you have used to measure the concept (e.g., Likert-scores and binary answers). So, I suggest you start by determining how many variables you want to create. For example, if your binary variables measure accuracy of knowledge about a topic, that is very different from opinions.
Another point to add is to conduct pre-testing of questionnaire first to ensure high quality translation before proceed to psychometric testing (reliability and validity assessment). Another source of reference for questionnaire validation is as below:
Streiner, D. L., Norman, G. R. & Cairney, J. (2015). Health Measurement Scales: A practical guide to their development and use. 5th ed. Oxford: Oxford University Press.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Applied Statistics
- Survey Design
- Questionnaire