I have assessed two models using Mplus. The difference in chi-square is approximately 10. How can I calculate the p value to know whether the latter model is significantly worse than the former one? Thank you
@Daniel,
Yes, they are. In the first model residual variances were not constrained and in the following model, they were. That is the only difference.
Witold
I would guess that an analysis of deviance in conjunction with a Chi-suqared test of the residual deviances will provide the significance value you are looking for.
I have no idea how this is achieved in Mplus, but in R you would have your generalized linear models (glm's) and compare them with
anova(glm1, glm2, test="Chisq")
If your case is more complicated and does not fit in this scheme you may still consider bootstrapping the null-distribution of delat-chi² values from which you can then determine the significance of your observed difference.
And a note at the end: (statistical!) significance is not a binary variable. The measure for significance (usually) is a p-value and this is on a continuous scale in (0, 1). The question "whether something is signifacant" is wrongly phrased. The correct question is "what (how large) is the significance". If the value obtained from your sample is large enough for you to decide that it is worth to prefer one model over the other... that decision needs your judgement call and is not autoatically given only from the data. In your case, for instance, using the "conventional" rule to base a decision upon by p
Hi Witold, just take the difference of the chi-square values of the nested models, calculate the difference of the degrees of freedom, and then look it up in the chi-square table. If the chi-square difference is not significant, then the more restricted model replicates the covariances just as well as the less restricted model. If the difference is significant, then the less restricted model fits the data better than the restricted model.
If you use a robust estimation method, such as MLR, you would need a different formula for calculating chi-square and df differences.
Regards, Karin
Hello Karen,
The estimator was ML. As I looked in the table, the difference in degrees of freedom of 5 'requires' a chi square difference of at least 11, and mine is 10 then I presume the latter model is NOT significantly worse than the former one. In this scenario then I am going to accept the more parsimonious model with more degrees of freedom. Hope I got it sorted.
Regards, Witold
http://courses.atlas.illinois.edu/spring2016/STAT/STAT200/pchisq.html
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size