I use the same Likert scale to assess 4 assessment scenarios, the scale is a combination of two existing models, so I thought of doing an EFA first and then CFA, but when I do that I get different loadings per scenario. How can I decide how to structure my model?
Hello Konstantina,
It is always possible that responses to your measure will vary as a result of scenario, such that the functional structure may vary (due to scenario). Prompt / scenario / stimulus differences have frequently been documented as a cause for different response profiles.
However, it seems that a simpler way to approach this question would be to evaluate the results from one scenario (via EFA, if you don't already have in mind a proposed measure structure). Then test the model-data fit of that model to the data sets for the other three scenarios. You may find that the structure you initially identified works satisfactorily. The fact that individual EFAs on each scenario yield different loadings isn't proof that "satisfactory fit" can't be obtained for a single model.
Good luck with your work.
Hello Konstantina,
It's normal to conduct EFA first and thereafter verify and test the model in CFA. It is also helpful to know that several models are possible - you probably won't find a perfect model. You have to consider both statistical and theoretical validity.
I would start by determining whether the scale is formative or reflective. Then you should choose an appropriate factor analytical method and assess reliability and validity. Can you tell us whether the scale is reflective or formative?
Good luck with your research.
Your question raised two questions for me.
Is the difference in loadings due to the scenarios, or is it the expected variation you get due to error variance as people complete the scale multiple times?
What is your sample size? The smaller the more variability you would expect in loadings as there will be more error variance.
Thank you all David Morse , Martin Brygger Andersen , Paul E. Spector for your answers! Martin Brygger Andersen is hard to say if is formative or reflective, the reason is that I use a well-tested model which normally has good reliabilities and correlations, but the other model has hardly been used, although theoretically, they relate to each other, thus I would say reflective. How I proceeded is to choose, based on theory and previous studies, the number of factors (4), I conducted a factor analysis using principal axis factoring and oblimin rotation. The reason is that theoretically the models I used match, but I was not sure how to structure the items per factor (so I used PAF), and I concluded they correlate (again based on theory and previous studies). I came to ok conclusions; the factors had not perfect, but good loadings and good reliabilities. Unfortunately, I have a sample of 218 participants, so I decided to not proceed with a CFA in this case, but I will mention that is needed if the scale is used in the future. Hopefully, this approach is good! If you have any recommendations, I would be happy to know!
- Badges
- Science topic
- Similar topics
- Methodology
- Research Methodology
- Research Papers