In my research, I have 11 multiple-choice questions about environmental knowledge, each question with one correct option, three incorrect options, and one "I don't know" option (5 options in total). When I coded my data into SPSS (1 for correct and 0 for incorrect responses) and ran a reliability analysis (Cronbach's Alpha), it was around 0,330. I also ran a KR20 analysis since the data is dichotomous but still not over 0,70.
These eleven questions have been used in previous research, and when I checked them, they all stated a reliability over 0,80 with a similar sampling to the sampling of my research. This got me thinking whether I was doing something wrong.
Low reliability might be caused by each question measuring knowledge from different environmental topics? If this is the case, do I still have to state its reliability when using the results in my study? For example, I can give correct and incorrect response percentages, calculate the sum points, etc.
Thank you!
You may ensure that:
- Correct responses are coded as 1 and incorrect responses as 0 during coding.
- If any questions are revised, reverse the coding accordingly.
- The content of all questions assesses the same aspect of environmental knowledge.
If still low, you may need to explore alternative methods for improving the reliability of the scale, such as revising or removing problematic items, conducting a factor analysis to identify underlying dimensions, or considering different scoring methods.
The questions taps into different aspects. I believe this is the reason of low reliability. But then, how to find Cronbach alpha?
If the questions tap into different topics (i.e., in case of multidimensionality), it likely does not make sense to apply a reliability measure such as Cronbach's alpha. Alpha implies a unidimensional scale/measurement model (i.e., all items measuring a single common factor).
Dear Christian, I appreciate for your response. That’s what I also think, I guess I am confused due to the past research.