I have produced a good-fitting, reliable baseline model according to CFA. The model contains several factors each loaded by several categorical (ordinal) items. I ultimately intend to analyze the factor scores across several grouping variables of interest.
As I understand it, the next step (Before analyzing factor scores) would be to ascertain configural, metric, and scalar invariance of the model across the groups of interest. This is to ensure any inferences made from subsequent analyses (such as ANOVAs) are attributable to true differences between the groups rather than the underlying model differing across the groups.
If the grouping variables take the form of a 2x2 factorial structure, which of the following options is most appropriate?
For example, say you have a 2 (A vs. B) x 2 (X vs. Y) design, do you:
A.) Conduct two sets of invariance tests, one for each of the grouping variables (e.g. one for the A vs. B grouping variable, another for the X vs. Y grouping variable)
B.) Create a new variable representing the 4 possible combinations of the two grouping variables (AX, AY, BX, BY), and conduct the invariance tests based on that
C.) ... Or is there another method for analyzing invariance in a 2x2 between-group scenario?
My intuition is that A will make it difficult or impossible to interpret any interaction I may find. But B may assume the 4 groups vary along a single dimension so I am unsure if I am violating underlying assumptions of the invariance test. Can anyone advise?
Secondarily, assuming invariance holds, any calculated factor scores should be based on this restricted model rather than the unrestricted baseline model, correct?
Hi Robert,
could you please be more specific about the design and what the 2x2 factors really mean?
I would end to use the four group model at once although I agree that this will make finding non-invariances a bit tedious. But that's a practical issue.
Depending what you plan to do from a substantial point of view, either metric invariance will be enough or scalar invariance (if you plan to do some sort of mean comparisons).
Best,
Holger
Holger Steinmetz Apologies, I should have clarified that the intention is to analyze individuals' factor scores using a 2x2 between subjects ANOVA.
For example:
2 (A vs. B) x 2 (X vs. Y)
A and B are mutually exclusive, X and Y are mutually exclusive. Meaning you have 4 sub-groups-
AX
AY
BX
BY
Pretty standard 2x2 design. So what I'm left with is wondering which of the following is most approprate:
-2 sets of invariance tests, one for each primary grouping variable: First (A, B) and second (X, Y)
-1 set of invariance tests across all subgroups (AX, AY, BX, BY)
And of course the secondary issue of whether, if invariance holds, you use the less restricted baseline model or the more restricted invariant model to calculate the factor scores.
Hi Robert, ok.
Asking for more context, however, meant *real* context instead of A, B, X, Y. Going more abstract does often not help (yeah, and I do it all the time, too :)))
Check the following for plausibility:
My favourite would be to go full blown latent and estimate latent mean differences in a four-group model.
For that you would strive for (partial) metric and scalar invariance--that is
a) testing the factor structure and striving for a clean fit (via chi-square), normally known as configural invariance
b) restricting all loadings to be equal (i.e., metric invariance). In fact of a misfit, allowing the problematic loadings to vary across groups. At the end, you should have two invariant loadings per latent variable (one of them being the marker item)
c) then restrict the item intercepts with the same procedure and outcome
d) Finally, restrict the latent means to strive for substantial group differences.
Here's a chapter and a paper that explain that in detail:
Steinmetz, H. (2010). Estimation and comparison of latent means across cultures. In P. Schmidt, J. Billiet, & E. Davidov (Eds.), Cross-cultural analysis: Methods and applications (pp. 85-116). Taylor & Francis Group: Routledge.
Steinmetz, H. (2013). Analyzing observed composite differences across groups: Is partial measurement invariance enough? Methodology, 9(1), 1-12. doi:10.1027/1614-2241/a000049
Factor scores, of course, are a practical solution that, still, incorporate error due to the factor indeterminacy.
Beauducel, A. (2005). How to describe the difference between factors and corresponding factor-score estimates. Methodology, 1(4), 143-158. doi:10.1027/1614-2241.1.4.143
There are some newer approaches to enhance them with information from the model, but as far as I know not really readily implemented (e.g., in Lavaan), see
Curran, P. J., Cole, V. T., Bauer, D. J., Rothenberg, W. A., & Hussong, A. M. (2018). Recovering predictor–criterion relations using covariate-informed factor score estimates. Structural Equation Modeling: A Multidisciplinary Journal, 1-16. doi:10.1080/10705511.2018.1473773
Devlieger, I., Mayer, A., & Rosseel, Y. (2016). Hypothesis testing using factor score regression: A comparison of four methods. Educational and Psychological Measurement, 76(5), 741-770. doi:10.1177/0013164415607618
Devlieger, I., & Rosseel, Y. (2017). Factor score path analysis: An alternative for SEM. Methodology, 13, 31-38. doi:10.1027/1614-2241/a000130
Devlieger, I., Talloen, W., & Rosseel, Y. (2019). New developments in factor score regression: Fit indices and a model comparison test. Educational and Psychological Measurement, 0013164419844552. doi:10.1177/0013164419844552
All the best,
--Holger
Hi Holger-
You've given me fantastic answers! Thanks greatly-
There is one thing I am still unsure of, however: which model is most appropriate to use as the basis for generating factor scores? The simpler baseline model or one of the more restricted, invariant models?
Cheers
Robert
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA