Probably a fundamental question to many of you esteemed statisticians here. But here is my case...
I found that some dependent variables that i am assessing (Y) does not have a significant relationship with the covariate (X). I understand that I should stop and only perform ANOVA or a Wilcoxon test; and NOT proceed with ANCOVA.
However, I just wonder what will happen if i proceed with ANCOVA and report the model anyways? Since the categorical-covariate interaction term is not significant, won't the model yield results similar to an ANOVA?
The reason i ask is because I am analysing several dependent variables. Some have significant covariate interaction while some does not. I am trying to keep the methodology (and its reporting) as concise and possible.
Thank you.
There is, in general, no great harm in including a non-significant predictor in the model. The exception would be if the non-significant predictor is highly correlated with the other predictor(s) in the model, especially if the sample is small; in that case (multi)collinearity could cause problems inestimating the model. As long as you are not in that situation, your reasons for including the non-significant covariate (consistency in modeling and reporting across all your DVs) makes good sense. If neither X nor the X*Z interaction term is significant in a model predicting Y from these two plus Z, then this full ANCOVA will indeed yield results for the effect of Z on Y that are very similar to what you would get with a simple ANOVA of Z alone predicting Y.
However, the way you have worded your concern is a little unclear. It is quite possible that the effect of X on Y is non-significant (with and/or without Z in the model), but becomes significant when the X*Z interaction term is included in the model. It is also quite possible for the X*Z interaction term to be significant in a model in which the X term is not significant. And of course, the X term could be insignificant without Z in the model, but significant when Z is included (with or without X*Z). If any of these possibilities is relevant to your concern, then by all means you should be reporting the full ANCOVA, with interaction, in order to accurately present the results. In such a case, you don't need to justify doing so by invoking consistency in reporting; it is essential to do so, statistically.
Thank you very much for your answer, Burke D. Grandjean! Clear and to the point!
Fully agree with dr. Grandjean.
If I may add anything to this discussion, I would pay your attention on "why we use statstical tests".
1. to test hypotheses
2. to explore the data if there is no or little hypotheses about the phenomenon.
Ad 1. In your case you should first ask a question about your hypothesis which is a foundation of your research. You should do that a prioiri, i.e. while planning your experiment. If this is your case, just test the planned hypotesis with an appropriate test. In some situations it is even recommended to add a priori adjustment to the model for e.g. sex and age as demographic variable, and you keep consequently the covariates, no matter they are significant or not.
Ad 2. If you are exploring the reality, you may test many different models (not only with or without interactions): both linear and non-linear, using multivariate approach as well. But if you report such results you should look at the results "with a pinch of salt" and report them adequately. Another way is to correct for multiple testing.
I hope this may help as well.
Good luck!
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics