If the ultimate goal of a measure developed through EFA is to correlate it with another measure within the same study, should I exclude the data from those who did not respond to the measure with which we are correlating the new measure? That is, if the ultimately goal is to correlate an old measure with the newly developed one, am I constrained to conducting an EFA with the data of participants who have responded to both measures?
First, a side issue. Is your goal to take a large number of variables, separate these into clusters of variables, and then use these to predict another variable, principal component regression might be useful. Still, if you like EFA more for some reason, use this, and then depending on the number of factors you might choose to use something like the lasso to look at the relationship between these and the other variable (and how is this other variable distributed ... I am assuming those you use with EFA meet those assumptions).
If want you EFA items to be based on the whole sample, use the sample. Ideally you would have a scale where enough psychometric work had been done to construct valid component estimates. Imagine an extreme example where you have 10,000 fill out the questionnaire and you create two scales from this. You then are interested, for example, in these two scales association with something observable in an expensive scanning machine, so have only ten subjects. You would not re-run the EFA only with this five.
Without delving much into the validity theory, we can say that you have, at least two goals, here.
One is to collect evidence regarding the factorial validity of your new measure. It is about answering the questions like "Does the relationship of my items with the latent factors make sense?", " Are they measuring the latent constructs that they suppose to measure" For this purpose, you may use the data from participants who have responded only to items of this scale.
On the other hand, you also may need to to provide some evidence on the "convergent validity" of your new scale which requires estimating correlations of your new scale with the other scales that measure the same construct. Usually, it is advised to acquire a new sample for this step, but I guess it is not an option for you. Then, for convergent validity, you can only use those participants who have responded to both scales.
But, if the factor structure and loadings vary dramatically depending on whether you included all or a portion of the participants, that may indicate a more serious problem about your scale in terms of its generalizability and measurement invariance.
I agree with both. However, in order to test for convergent validity, you really need to use only the those participants who have responded to both scales.
- Badges
- Science topic
- Similar topics
- Instrumentation Engineering
- Instruments