You have a lot of questions and that is good. EFA, CFA, and SEM should be done in that order on your data (not as separate techniques). In grad school I was taught that EFA should have a sample size 15-20 times the number of items (see Evaluating the use of exploratory factor analysis in psychological research by LR Fabrigar, DT Wegener, RC MacCallum, EJ Strahan available for download on ResearchGate).

CFA and SEM are affected by sample size. Here is a frequently-cited article on the subject: Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size by Marsh, Herbert W.; Balla, John R.; McDonald, Roderick P., Psychological Bulletin, Vol 103(3), May 1988, 391-410. doi: 10.1037/0033-2909.103.3.391. The CFA goodness of fit statistics, such as chi-square, are typically significant (and sometimes ignored) for very large sample sizes.

I would highly recommend James Gaskin's YouTube videos on data screening, EFA, CFA, and SEM. An integrated view of these procedures is advanced that I found to be quite helpful.

As to your bonus question, you assumption of data equivalence seems plausible. You might use some parametric tests to examine similarities/differences of the cohorts.

I have simply "scratched the surface" so to speak. Good luck with your dissertation research!

Hello, Nicholas!

Thank you very much for your answer and kind wishes! I will definitely read the papers you recommended.

While your information on sample sizes is very useful, I was more concerned about the optimal strategy for dividing the sample between pilot and main research when comparing alternative models as well as CB and PLS approaches (does the same logic apply here, too: pilot analysis for each alternative model (or SEM approach), then modification of indices, if needed, then main analysis for each model (or SEM approach).

In regard to James Gaskin's YouTube videos, I have watched some of them and indeed find them very helpful with very clear explanations. Having said that, they lack some methodological aspects, hence my questions.

You write: "EFA, CFA, and SEM should be done in that order on your data (not as separate techniques)". Could you clarify what do you mean here by "not as a separate techniques"?

Kind regards,

Aleksandr

Hi Aleksandr,

I think as with every good question, there are multiple good answers. I have seen exploratory and confirmatory data sets divided by time order (i.e., data set A was collected before data set B), and I have seen multiple data sets combined and the exploratory data set was randomly selected from the complete data set. Both options are typically okay with reviewers and ideally would produce the same results (if your factor structure is true for the population from which both datasets were drawn from). If you are really worried about this sampling strategy, I would suggest going with the one that you are most comfortable with (random sampling??) and then using the other option and adding a foot note indicating that the results were the same.

For the other questions, as long as you run the EFA separate, the CFA and SEM can be run on the same data set. In fact, you could allow all of the factors to covary on the CFA and then use that model as a nested comparison for your SEM model(s). That is a part of the 2-step SEM process.

I hope that this is at least somewhat helpful! As Dr. Beutell mentioned, this is only scratching the surface, researchers have made their careers over the nuances of when and how to use SEM.

Good luck with your project!

-Xander

Hi, Xander!

Thank you for kind wishes! Your reply is indeed helpful and is much appreciated.

Best regards,

Aleksandr