My study involves 2 groups (Black and Latinx participants, N=816). I've run my structural equation models with the data all together and will get a completely different chi-square when I specify the 2 groups in AMOS. I do this using the "manage groups" and "select data" commands. When I run it, it doesn't give me separate fit statistics for each group, so I'm not sure I understand why the exact same data would have different results just because I labeled the groups. Also, which chi-square value should I report for overall fit?
I appreciate any help I can get understanding this.
It makes sense that you get a different chi-square when the model is fit to each group separately. The overall model for the whole sample implies that your population is homogeneous (i.e., that the parameters of the model apply to everyone), whereas the multigroup analysis allows certain (or all) parameters to vary across groups. The overall ("all together") model with the same parameters in both groups makes sense only if you can show through a multigroup analysis that all model parameters are equal ("invariant") across the two groups. With "all parameters" I mean all
- loadings
- intercepts
- error variances
- factor variances
- factor covariances
- factor means
We typically start with a configural invariance model (equal factor structure and loadings patterns, but no parameter equality constraints across groups). If that model fits well, we specify equal loadings across groups (weak invariance), then also equal intercepts (strong invariance), then also equal error variances (strict invariance). With at least weak invariance, you can test for equal factor variances and covariances also. With at least strong invariance, you can also test for latent mean differences.
If a model with all parameters set equal across groups fits well, you could go without a multigroup model. In that case, there is no difference whatsoever between groups and your "all together" model would be OK to use.
It makes sense that you get a different chi-square when the model is fit to each group separately. The overall model for the whole sample implies that your population is homogeneous (i.e., that the parameters of the model apply to everyone), whereas the multigroup analysis allows certain (or all) parameters to vary across groups. The overall ("all together") model with the same parameters in both groups makes sense only if you can show through a multigroup analysis that all model parameters are equal ("invariant") across the two groups. With "all parameters" I mean all
- loadings
- intercepts
- error variances
- factor variances
- factor covariances
- factor means
We typically start with a configural invariance model (equal factor structure and loadings patterns, but no parameter equality constraints across groups). If that model fits well, we specify equal loadings across groups (weak invariance), then also equal intercepts (strong invariance), then also equal error variances (strict invariance). With at least weak invariance, you can test for equal factor variances and covariances also. With at least strong invariance, you can also test for latent mean differences.
If a model with all parameters set equal across groups fits well, you could go without a multigroup model. In that case, there is no difference whatsoever between groups and your "all together" model would be OK to use.
If I'm understanding what you've done correctly, you can subtract the two chi-square values, and also the two df values, to get a new chi-square and df that tests whether allowing the parameter estimates to differ between your two groups gives significantly better fit to the data than constraining both groups to have a single set parameter estimates in common.
If they don't differ significantly, especially if not close to significance, your "all together" model is preferable because it's more parsimonious. If they differ significantly, especially if the parameter estimates are also substantively different between groups (with reversals of sign or large differences in magnitude), then your "separate groups" model is preferable, despite being more complex.
Model fit indices would obviously change. As soon as you specify another group (using multiple group method) in the AMOS by specifying group variable, the data file would be split into parts based on the number of categories of the grouping variable.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Techniques
- Structural Equation Modeling