I have fit a random intercept factor model (Maydeu-Olivares & Coffman, 2006) in Mplus by the following syntax. One substantial factor where all item loadings are freely estimated and one random intercept factor where all item loadings are constrained to be 1. The random intercept factor variance is estimated.
Syntax:
ANALYSIS: ESTIMATOR = MLR;
ROTATION = GEOMIN(ORTHOGONAL);
MODEL: G BY a-a20 (*1); # G represents substantial factor
RI BY a-a@1; # RI represents random intercept/method factor
RI with G@0;
G@1;
But I got output as following. It looks like that the algorithm automatically takes G as the method factor because the factor loadings from this factor are much smaller than RI and so does its variance (if I delete G@1 to let the variance be estimated too). What I want the algorithm does is to take G as the substantial factor and estimate its factor loadings and take RI as the method factor and only estimate its variance. Does anybody know how to specify this in Mplus? Thanks a lot!
MODEL RESULTS
Two-Tailed
Estimate S.E. Est./S.E. P-Value
G BY
A1 0.445 0.118 3.771 0.000
A2 -0.085 0.137 -0.624 0.533
A3 0.148 0.111 1.334 0.182
A4 0.106 0.118 0.898 0.369
A5 0.184 0.162 1.137 0.255
A6 0.239 0.095 2.523 0.012
A7 0.444 0.125 3.556 0.000
A8 0.193 0.093 2.077 0.038
A9 -0.008 0.084 -0.092 0.926
A10 0.395 0.133 2.972 0.003
A11 0.094 0.166 0.563 0.574
A12 0.258 0.137 1.879 0.060
A13 0.318 0.117 2.717 0.007
A14 0.712 0.091 7.819 0.000
A15 0.316 0.104 3.027 0.002
A16 0.504 0.129 3.911 0.000
A17 0.290 0.104 2.786 0.005
A18 0.957 0.086 11.064 0.000
A19 0.485 0.164 2.951 0.003
A20 0.484 0.122 3.974 0.000
RI BY
A1 1.000 0.000 999.000 999.000
A2 1.000 0.000 999.000 999.000
A3 1.000 0.000 999.000 999.000
A4 1.000 0.000 999.000 999.000
A5 1.000 0.000 999.000 999.000
A6 1.000 0.000 999.000 999.000
A7 1.000 0.000 999.000 999.000
A8 1.000 0.000 999.000 999.000
A9 1.000 0.000 999.000 999.000
A10 1.000 0.000 999.000 999.000
A11 1.000 0.000 999.000 999.000
A12 1.000 0.000 999.000 999.000
A13 1.000 0.000 999.000 999.000
A14 1.000 0.000 999.000 999.000
A15 1.000 0.000 999.000 999.000
A16 1.000 0.000 999.000 999.000
A17 1.000 0.000 999.000 999.000
A18 1.000 0.000 999.000 999.000
A19 1.000 0.000 999.000 999.000
A20 1.000 0.000 999.000 999.000
RI WITH
G 0.000 0.000 999.000 999.000
Intercepts
A1 3.251 0.096 33.728 0.000
A2 3.246 0.098 32.988 0.000
A3 2.071 0.079 26.213 0.000
A4 1.976 0.083 23.804 0.000
A5 2.915 0.107 27.167 0.000
A6 1.773 0.069 25.742 0.000
A7 2.668 0.091 29.478 0.000
A8 2.431 0.077 31.708 0.000
A9 1.498 0.066 22.761 0.000
A10 2.545 0.092 27.682 0.000
A11 2.910 0.101 28.903 0.000
A12 2.720 0.098 27.665 0.000
A13 1.891 0.079 23.824 0.000
A14 2.346 0.083 28.217 0.000
A15 1.739 0.070 24.819 0.000
A16 3.057 0.091 33.654 0.000
A17 2.066 0.072 28.682 0.000
A18 2.445 0.089 27.498 0.000
A19 3.289 0.113 29.187 0.000
A20 2.654 0.098 27.014 0.000
Variances
G 1.000 0.000 999.000 999.000
RI 0.492 0.073 6.751 0.000
Reference: Maydeu-Olivares, A., & Coffman, D. L. (2006). Random intercept item factor analysis. Psychological methods, 11(4), 344.
My first question would be: Why do you think you need a method factor? What kind of a method effect is this factor supposed to represent (e.g., the effect caused by using negatively vs. positively worded items?)?
Many loadings on your target (substantive) factor are not significant, indicating that the model may be misspecified and/or overparameterized and that the target factor is not easily interpretable. In Geiser, Eid, and Nussbeck (2008) we provided a critique of the random intercept factor model and presented an alternative approach for handling method effects that can be used to deal with item wording and other method effects in a more theory-driven way (e.g., by specifying a residual method factor only for the negatively worded items), in which both the target and the method factor have a clear a priori psychometric meaning and interpretation. See:
Article On the Meaning of the Latent Variables in the CT-C (M-1) Mod...
An additional issue in your application may be that you have a large number of items (20). The items may not be unidimensional even after accounting for wording or other method effects. There may be additional substantive factors (a single substantive factor may not be enough).
@Christian Geiser Thanks a lot for your thoughtful answer and sorry that I didn’t mention the reason for choosing the random intercept approach!
Reason 1:
I choose random intercept mode because in this case all the twenty items are in the same keying direction, where CT-C(M-1) approach is not applicable as far as I understand. But if it is incorrect, please point me out! And I'm not sure whether restricted CT-C (M-1) can be extended to multidimensional occasions?
Reason 2:
Based on my personal experience. I do agree with you that the underlying assumption of the random intercept approach is too strong: the existence of individual differences in scale usage (e.g., response style) but these differences only exist between respondents and keep constant across items within respondents (because all item loadings of the random intercept method factor need to be set up as 1 or constrained to be equal).
But I have privately explored different (E)SEM approaches (CT-C(M-1), random intercept, exploratory structural equation modeling) used to account for method effects when developing a new scale, I (maybe only intuitively) found that:
a) using any approach will significantly improve fit indexes and do clarify the factor structure compared with using conventional EFA;
b) the differences in the factor loadings and conclusions of using different approaches are negligible, maybe it is because my scale actually turned out unidimensional according to all approaches. But when someone is exploring a more complicated factor structure, using different approaches would lead to different results and conclusions);
c) most importantly, this is the second reason why I choose the random intercept approach (at least in this case because I choose ESEM when I explore the assumed complex structure of the new scale). I found that the validities of the method factors are more or less different. By mentioning validity, I personally mean the correlations of the factor scores of the method factor to other constructs. If the approach is valid to only account for method effects, then correlations of the factor score of the corresponding method factor to substantial constructs should be as small as possible. If the scores of substantial constructs also derive from the same approach, their correlations should be as close to zero as possible. I tried to figure out this with the data on my hand but the scores of substantial constructs are only sum scores for convenience… and I found that only the factor score of the method factor of the random intercept model has the smallest correlations to other constructs (below .10), factor scores of the method factors by using other approaches would significantly correlate with some constructs (and all the method factor scores almost perfectly correlated with each other, ranging from .66 to .99). Although the magnitudes of the correlations are quite small (usually below .20), they indicate that method factors of other approaches more or less unintentionally account for variance unrelated to method effects.
Again, I do agree with you that the assumption of the random intercept approach may be too strong and a bit unrealistic, but my data implied that the variance accounted for by the method factor of this approach is more ‘pure’, which is shown by the negligible correlations with other constructs.
Bo Wang : Thanks for your reply! This is an interesting topic.
My personal problem with the random intercept factor model is that in this model neither the target (substantive) factor nor the random intercept factor have a clear psychometric definition and meaning, especially when many items do not load significantly on the target factor. It is then very difficult to figure out what is actually represented by each factor. This seems to be the case in your application as well where the loading pattern is inconsistent and you have to rely on external correlates to figure out what each factor may represent (substantive variance? method variance?). I prefer measurement models in which all factors have a clear definition, meaning, and interpretation so these issues can avoided. Like you wrote, models like the random intercept model can always improve model fit, but that does not mean that they provide meaningful latent variables or that they help you better understand your measures and constructs.
In this case, I'd say I'd probably prefer a simple-structure factor model with multiple correlated factors (if possible) to reflect the multidimensionality of the items. Such simple-structure measurement models with correlated factors are often easier to interpret than models that contain what I would consider "obscure" method factors. Have you tried an exploratory factor analysis to see whether you can identify multiple substantive factors or subfacets?
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Techniques
- Structural Equation Modeling