We recently develop a game call "Palaeo", it's a game that can help kids to understand an ancient elephant in a fun and engaging way.
We have collected some feedback using the IMI questionnaire, but after I run CFA, there are some variables that was originally in "Interest/Enjoyment" and "Effort/Importance" factors are mixed together into one factor, and there's one variable in "Interest/Enjoyment" factor mixed into "Value/Usefulness" factor.
I don't know why it happen, I only change some part of the questions so it's about playing "Palaeo", other then that it's the same with the original IMI questionnaire and the other factors are completely fine. I guess it's because the questionnaire was translate into Chinese, some word might have different meaning in English ? Or maybe it's because we conduct our research only on 51 elementary school students?
What should I do with this? Please tell me your solution.
Thank you, I appreciate your help.
These are the two factors that mix up together, no 6, 9, 10 got deleted because the loading is less than 0.5, no 5, 8, 11, 12 are mix into new factor and no 7 is mix into "Value/Usefulness" factor.
(Interest/Enjoyment)
5. I think Palaeo is fun.
6. I enjoyed playing Palaeo.
7. Palaeo was interesting to play.
8. My full attention was on Palaeo while i was immersed.
(Effort/Importance)
9. I put a lot of effort into playing Palaeo.
10. I tried very hard in Palaeo.
11. I did my best in Palaeo.
12. I tried very hard to do well in Palaeo.
N = 51 is indeed a rather small sample for a CFA. This may explain some of your unexpected results.
N=51 is considered small sample size for CFA. CFA is sensitive to sample size. It reduces the reliability and stability of the factor solution. Since you have 12 items in your IMI sections, the ideal would be at least ~120 (min 10 per items). N=21 would lead to falsely accepting/rejecting factor relationships.
I also personally think that it could also be due to the translation and the age of your respondent. Since your respondent is elementary students, they might not clearly differentiate between effort and interest as their cognitive development and understanding of abstract constructs is still growing. Hence they tend to interpret scales less precisely than adults or teens.
There's not much you can do since it's your population's response. You may want to be cautious when interpreting the IMI factors, or use the emergent factors and name/interpret them accordingly. If you suspect that it may be due to the translation, consider doing a back-translation? Have someone translate the Chinese questionnaire back to English and see if the original meaning still survive or if the questions are confusing to the student.
You can try to do EFA and see if you could refine your model? If you keep using the original factor labels, you're misrepresenting how your participants actually responded. EFA tells you the real factor structure for your group of students. You can analyze and report using those new factors instead of sticking to the original IMI ones that don’t fit.
Best wishes.
- Badges
- Science topic
- Similar topics
- Linguistics
- Composition Studies
- Publication