I am modeling a multivariate multilevel model (level 1: response (4 different responses; cross-sectional design); level 2: pupils (N = 3997); level 3: schools). For each predictor I have added separate coefficients for each response (interactions).
Some of my higher-level predictor variables (school characteristics) do not have a significant effect on all the response variables. Is it best to keep the non-significant interactions in the model or to exclude them?
I would say that it is worth running the model removing the non-significant variables one by one, to check whether the non-significance is due to collinearity (that is, 2 variables might be significant when included in the model on their own, but both insignificant when included together - perhaps because they are measuring the same theoretical construct.
But when removing variables don't then forget that they ever existed - make sure you discuss and interpret their non-significance. What theories does the non-significance support/reject?
[NB this answer would apply to any regression model, not just MLM]
I would say that it is worth running the model removing the non-significant variables one by one, to check whether the non-significance is due to collinearity (that is, 2 variables might be significant when included in the model on their own, but both insignificant when included together - perhaps because they are measuring the same theoretical construct.
But when removing variables don't then forget that they ever existed - make sure you discuss and interpret their non-significance. What theories does the non-significance support/reject?
[NB this answer would apply to any regression model, not just MLM]
I would also asses the adjusted coefficient of determination upon the removal of non-significant variable if it does not deteriorate go ahead and remove those variable. Also estimate Variance inflation factors (VIF) for all the variable to see if they are correlated in any ways.
If you have nested data (level 1 & 2), you may need to move your to a MLM analyses to properly assess effects of your predictors.
Hope this helps,
Matt
I recommend to you to keep the non-significant interaction, because non-significant does not mean inexistent. Non-significant means that your experimental data do not provide enough evidence for declare the effect significant. The main reason is that the magnitude of the non-significant effect is in the same magnitude order than experimental error, therefore the apparent effect may be due to experimental dispersion.
- Badges
- Science topic
- Similar topics
- Teacher Education
- School