Eric Orr Performing EFA and CFA on the same dataset serves no use. EFA is used to extract factors for the first time from a dataset, whereas CFA is used to verify factors extracted from a separate dataset.

However, exploratory factor analysis (EFA) is commonly employed to determine a measure's factor structure and to assess its internal reliability. When researchers have no theories regarding the nature of the underlying factor structure of their measure, EFA is frequently advised.

Hello Eric,

So, your judgment is that the four-factor EFA you decided upon does not make sense? If you had in mind a specific factor structure for the measure and the five-factor model was, in fact, the hypothesized solution according to theory, prior research, or just your educated guess prior to looking at any results, then:

You likely would have been better off running a CFA on the target model, and abandoning it only if the results showed exceedingly poor model-data fit which couldn't be satisfactorily ameliorated via benign modifications (e.g., allowing for a pair of variables loading on the same factor to have a shared covariance/correlation).

If no a priori model was hypothesized, then yes, you'd need to rely on EFA as your guide. However, there are many ways in which two persons analyzing the same data set could wind up declaring different factor structures as the "best choice."

For me, the ideal solution for overcoming your dilemma as to what's going on with the structure of your measure would be to: (a) collect additional data; and (b) evaluate, via CFA, both the four- and five-factor solution; then (c) determine which, if either, seems to work more satisfactorily for your research needs.

Finally, I think it's pretty darn clever of BYU to have a Hawai'i campus.

Good luck with your study.

Hello all,

beyond what David said: Perhaps you have variables in your dataset that should be differently related to the two factors in question? I am, however, skeptical about the value of simply collecting new data and repeating the same exercise as I doubt that the most substantial danger of exploratory approaches (I would perceive your CFA as semi-exploratory) is capitalization on chance (or overfitting) but a systematic misspecification of the model structure. The latter would not be revealed by new data but by incorporating new variables that add new testable constraints (and estimated relationships that help to evaluate the validity of any factor solution beyond the fit).

As an example see the paper in which I was allowed (that is an exception) by the journal to re-specify a misfitting model. It is an exception as most journals would require either to keep the misfitting model and to represent it as "ok" or present incentives to cheat (that is to tinker with the model and to represent it as the result of the test). This is part of the replication crisis that is not yet discussed in the scientific community.

Rosman, T., Kerwer, M., Steinmetz, H., Chasiotis, A., Wedderhoff, O., Betsch, C., & Bosnjak, M. (2021). Will COVID‐19‐related economic worries superimpose health worries, reducing nonpharmaceutical intervention acceptance in Germany? A prospective pre‐registered study. International Journal of Psychology, 56(4), 607-622.

All the best

Holger

Hello! I am a student but i've been battling the covariance matrices for some time, so here are my 2 cents.

Maybe ur problem arises from how you decide on how many factors to extract. If you rely on eigenvalue>1, this could be the problem. Sometimes it overextracts, sometimes it underextracts. Like all methods its not the penultimate way of solving this problem. Have you tried running a parallel analysis to contrast its results with the standard scree plot and the eigenvalue>1 rule? Just an hour ago, I was extracting factors on an adaptation of an a priori 2-factor questionnaire im working on, and eigenvalue>1 rule underextracted just because the second factor had an eigenvalue of .991 or something. Parallel analysis clearly showed 2 extractable factors.