I have to validate the scale I used in my research. I used the same scale comprising 14 items in 4 independent scenarios (that were presented to participants with no specific order and no difference in time, the one after the other). I have a sample of 218. What I thought to do is an EFA with PAF and Varivax rotation for each assessment scenario. I got feedback that I should perform an EFA in all the data sets at once (all 4 scenarios, so in all 56 items), but I doubt this. What should I do (concerning that my sample is only 218 and I cannot do an EFA and a CFA in the same sample)? Also to mention that my sample consists of 3 unequal groups.
Hello Konstantina Iliadou
first of all, ignore the spam above my post. I really hope posting unrelated articles does not become a rising trend on RG. I find this extremely rude.
As to your questions, you cannot "validate" any factor structure with EFA - varimax is another issues, but it is also not recommended. Furthermore, your explanation is conflicting: If you obtained responses from 218 people on the same 14 items, administered under similar conditions (e. g. same order of items), you should treat them as one big sample for which scalar measurement invariance should be present. However, I do not know what you mean with "3 unequal groups".
Please provide more information on your goal and design.
Best
Marcel
Thank you for your answer Marcel Grieger . What I mean by 3 unequal groups is that I have 3 groups of participants, 30 of them are trainers, 33 general population, and 141 are test candidates (maybe too much information, sorry for that). When I perform an EFA treating the data as a big sample (so all the 218 participants and the 56 items (14 per scenario)) I get perfect loading of the items per scenario (without cross-loading). What I understand is that due to my small sample size for the 2 groups of participants, I cannot perform a Multigroup confirmatory factor analysis. But my question is, that based on the theory and the models I used, I have 4 factors (perceptions) that I should somehow validate, should I, as you mentioned, perform an EFA in all datasets or should I split my sample into 2 (I was thinking 50 and 168) and use the first 50 for an EFA in one scenario and the rest 168 for CFA in all four scenarios? Thank you in advance.
Hey Konstantina Iliadou
there is no such thing as too much information when it comes to these matters - well, sometimes there is, but nevermind. Indeed, MG-CFA seems out of the question due to the unequal and small sample size. I would still opt for CFA although be it without controlling for invariance. Splitting your sample is generally not recommended either, as it can bias your estimates severely.
I still fail to understand how you arrive at 56 and not 14 items. What are those "four scenarios" you keep mentioning? Perhaps a look at the EFA you have already run would help me understand.
Best
Marcel
Hi Marcel Grieger
The same survey of 14 items was used to measure the receptiveness of the participants to 4 different assessment scenarios, one at a time, following by each other and randomly presented to the participants, meaning that in total, the questions/items were 56 for all assessment scenarios. I hope is clearer now. What I decided to do is to randomly select one of the four scenarios and conduct an EFA, and after I made sure the model structure fits with the theoretical model, I performed 4 CFAs for each assessment scenario. I indeed split the sample into 2, I didn't have any other option since this is my thesis project, and I am not going to collect more data.
Hi Konstantina Iliadou
now I understand how you arrive at 56 items. Nevertheless, I am left slightly puzzled on how to proceed. Ideally, you would want your 14 items to work under all four scenarios and in all three groups. I am afraid, I cannot wrap my head around this - also for a distinct lack of time at the moment.
Wishing you the best of luck. Perhaps someone else may be able to helpt you further.
Marcel
- Badges
- Science topic