The reviewer of a research paper has sent me this response:
the methodology that you have used in building measurement instruments is not quite correct. It seems that you haven’t use measurement scales from the previous researches instead of that You have used selected items from the other researches to build new measurement scales. To test that kind of instruments it is not enough to use EFA than CFA on the same sample.
I am not sure what to do if EFA and CFA are not the right tests to establish the validity of my instrument (questionnaire). Please help what the reviewer is asking me to do. Thanks!
In accordance with the comments of reviewer it seems that you have taken different items from different scales thats why the reviewer has asked for EFA and CFA. As per my knowledge if it is the case that you have even adopted different questions from different scales (which are already validated) still you have to run both tests. Because you may have used questions according to your own study need from different places. so you have to first run EFA to let dimensions of scale to be known and then run CFA to confirm the results that questionnaire that you have used is valid for your study and can measure the variables under investigation.
Combining scales items from different scales is primarily not a good idea. unless you have strong theoretical justification for that. lets assume there was a need to do that, then the combined items should be considered as initial pool of items which required content validity. Thus, in my view the first step after justifying the need of combining items was to go for content validity. once the process is followed, you need to split the data into two parts and use one part to test your scale through CFA and second part to validate the results. The reviewer might be asking to provide a table containing results of correlations, construct reliability and average variance extracted. Hope this helps.
Hi Kashif,
as the answers above imply, the quote
"You have used selected items from the other researches to build new measurement scales. To test that kind of instruments it is not enough to use EFA than CFA on the same sample"
is not telling enough to answer the question what to do. In fact, it is simply true that you cannot use a CFA after aplying a EFA on the same data. But this was pointed out correctly above. "EFA than CFA" by the way is hopefully a typo and should mean "then" implying a sequential analysis.
Anyway, it seems that you had a rationale in selecting items from larger scales (I hope at least that 'from researchers' means 'from established measurement scales'). One good argument for that would be, that they load high on the assumed underlying factors. In this case reducing the set of items is quite reasonable when over-identification of a latent factor in CFA could be a problem. I also do not think, that the reduction of the item set-size (number of indicators) means that they have to be explored regarding their factor structure unless it is the goal of the study (in the latter case see comments above). In other words, when you want to test a hypothesis regarding the factor structure and have reason to think that reducing the number of factor indicators does not change the hypothesized factor structure, then just conduct CFA. However, when you mix items from different scales, or add additional items, then EFA should be carried out first, either in a first expirment, or in a split-sample scenario (again see above).
When the reviewer says "it is not enough" then he might also think of scale and item reliability analyses, because a factor can only load as high on a variable, as the variable reliability allows (measurement error might be a problem detecting factors).
Hope this helps, but I think for more detailed responses you may lay out some more details on how you built your scales, and the goal of your study.
Best, René
- Badges
- Science topic
- Similar topics
- Methodology
- Research Methodology