Dear all,
I am eager to conduct power analysis by employing the Monte Carlo simulation available in Mplus. I have run a Multivariate Latent Growth Curve model (MLGCM) with my three-wave longitudinal data in my study, and thus, I have followed the guideline presented by Muthen & Muthen (2002; Article How to Use a Monte Carlo Study to Decide on Sample Size and ...
). However, I guess, I am a systematic mistake (see, the attached output of monte carlo simulation for % sig. coefficient for mean score for SNAT = .134) but, unfortunately, I cannot find out what the problem/mistake is. Is there anyone who might help out to find the mistake(s)?The out of the Monte Carlo simulation is attached. Besides, you can see the output of my MLGCM below. Thanks in advance for any kind of help!
MODEL RESULTS
Two-Tailed
Estimate S.E. Est./S.E. P-Value
INAT |
ID_T1 1.000 0.000 999.000 999.000
ID_T2 1.000 0.000 999.000 999.000
ID_T3 1.000 0.000 999.000 999.000
SNAT |
ID_T1 0.000 0.000 999.000 999.000
ID_T2 1.000 0.000 999.000 999.000
ID_T3 2.000 0.000 999.000 999.000
IET |
ID_G1 1.000 0.000 999.000 999.000
ID_G2 1.000 0.000 999.000 999.000
ID_G3 1.000 0.000 999.000 999.000
SET |
ID_G1 0.000 0.000 999.000 999.000
ID_G2 1.000 0.000 999.000 999.000
ID_G3 2.000 0.000 999.000 999.000
SNAT WITH
INAT -0.083 0.041 -2.037 0.042
IET WITH
INAT -0.208 0.080 -2.582 0.010
SNAT 0.019 0.034 0.556 0.578
SET WITH
INAT 0.063 0.037 1.732 0.083
SNAT 0.017 0.014 1.178 0.239
IET -0.205 0.087 -2.361 0.018
Means
INAT 3.497 0.061 57.275 0.000
SNAT -0.027 0.026 -1.018 0.309
IET 3.479 0.078 44.763 0.000
SET -0.129 0.035 -3.713 0.000
Intercepts
ID_T1 0.000 0.000 999.000 999.000
ID_T2 0.000 0.000 999.000 999.000
ID_T3 0.000 0.000 999.000 999.000
ID_G1 0.000 0.000 999.000 999.000
ID_G2 0.000 0.000 999.000 999.000
ID_G3 0.000 0.000 999.000 999.000
Variances
INAT 0.706 0.092 7.662 0.000
SNAT 0.079 0.031 2.568 0.010
IET 1.141 0.156 7.316 0.000
SET 0.148 0.069 2.135 0.033
Residual Variances
ID_T1 0.131 0.068 1.929 0.054
ID_T2 0.244 0.040 6.055 0.000
ID_T3 0.057 0.063 0.901 0.368
ID_G1 0.115 0.116 0.998 0.318
ID_G2 0.338 0.059 5.696 0.000
ID_G3 0.139 0.105 1.324 0.186
As far as I can see, the slope factor mean in question is set to a value of -0.02 in the population model:
ESTIMATES S. E. M. S. E. 95% % Sig
Population Average Std. Dev. Average Cover Coeff
Means
SNAT -0.020 -0.0200 0.0246 0.0244 0.0006 0.950 0.134
The parameter is thus rather close to zero and you may need a larger sample (> 250) for sufficient power to detect that it is different from zero. With N = 250, your power is estimated to be only .134. This seems like a plausible result to me.
Did you intend to set this slope factor mean to such a low value in the data-generating population model? This value indicates that there is almost no mean change over time. This may or may not be what you want. If there isn't any mean change in the population (population slope mean = 0), then the column "%sig coeff" for the mean parameter gives the Type-1 error rate rather than power. This may be fine--depending on your assumed population values.
I noticed that you seem to have quite a few replications with warning messages about non-positive definite matrices (potential improper solutions with negative residual variances etc.; see TECH9 output at the end of your file). This is something to look into as well. The sample size of N = 250 may not be large enough to consistently obtain proper solutions.
Dear Prof. Geiser,
First of all, many thanks for your pretty clear answer.
Indeed, -0.020 is the slope factor mean in my original model. In fact, theoretically speaking such a stable pattern makes more sense rather than a relatively increased or decreased pattern. As you indicated, %sig coeff might indicate Type-1 error. I will closely monitor the warning messages as well.
Thanks again!
Best,
Savaş
- Badges
- Science topic
- Similar topics
- Engineering
- Control Theory
- Control Systems
- Control Systems Engineering