Jochen

In your example grp and age is highly correlated and therefore removing age while retaining grp in the model increases significance of grp. Moreover sex is not very related to outcome, or one cay the effect of sex disappears when grp is included in the model.

Also by prediction it is generally meant the ability of model correctly guessing new outcome values. I guess you meant "estimation" when using the word "prediction" because I can not see anything related with prediction in your answer.

I would not look for the significance of the covariate itself. I would look for the impact of including the cofactor on the prediction (accuracy, precision) of the response variable. It may be that the p-value for a covariate (or a set of several covariates as a group!) is rather large but it yet has a relevant impact on the prediction of the response. It may also be that covariates with very low p-values do not have any relevant impact on the prediction of the response (I would exclude such predictors despite their "statistical significance" to get a simpler model that does its job still sufficiently well).

PS: be careful with analysing "scores" - they are often not interval scaled, and therefore models estimating differences between scores can give a very misleading impression.

Dear Shuran,

your reasoning is correct. it only makes sense to include control variables (the ariates in ANOVA) if they have some influence in your model

Lisete, yes, you are right that they should have "some influence" - but the relevant influence is not measured by the statistical significance of the estimated coefficient for this variable or of the estimated reduction of the residual variance but rather by the impact on the prediction of the response. It can be disadvantagous (not to say "stupid") to keep variables with low p-values and also to remove variables with high p-values.

I will give an example:

grp = (0, 0, 0, 0, 1, 1, 1, 1)

age = (12,13,14,13, 78,45,63,72)

sex = (0, 1, 0, 1, 0, 1, 0, 1)

y = (106.07, 107.67, 106.77, 107.77, 141.46, 125.73, 132.68, 138.95)

The variable grp is the interesting treatment variable, age and sex are possible covariates. The age is deliberately confounded with the treatment.

The full model shows nice small p-values for age (

Dear Jochen,

Thanks fou your reasoning and your explanation. Very helpful!

Jochen

In your example grp and age is highly correlated and therefore removing age while retaining grp in the model increases significance of grp. Moreover sex is not very related to outcome, or one cay the effect of sex disappears when grp is included in the model.

Also by prediction it is generally meant the ability of model correctly guessing new outcome values. I guess you meant "estimation" when using the word "prediction" because I can not see anything related with prediction in your answer.

Dear Dr. Wilhelm, 

Thank you for your answers and the example! I get your point that cannot decide if to include a variable as covariate in the model only by looking at the statistical significance of this variable, but to look for the impact of including this variable on the prediction of the response variable. But I'm not sure how to decide the impact on the prediction of the response (I have some difficulty to follow your example). I can hardly predict which brain regions will show in a whole brain analysis (an exploratory method) in SPM. I have tested and found that results vary quite a bit when including different covariates. So I need to decide which are "correct" covariates to include based on a principle to get reliable results. 

May any SPM experts help to answer?

Mehmet,

yes, I know about the correlation. That was intended to get a simple-to-understand example. I also pointed this out in my post above.

Regarding prediction/estimation: you are right, I looked at the coefficients, which are estimates of effect sizes. I did not look for the prediction of the response (outcome) variable. I was not using the right words - sorry!

Checking the accuracy of predictions is surely a different thing. It may be more relevant to look for prediction than to look for estimation (of coefficients) - that depends on the purpose of the model. However, if prediction is the relevant task, then one would need to apply some (cross-)validation procedures to ge a measure for the performance, and this was too much a demand for a small (possibly not very realistic, but hopefully yet instructive) example.

Anyway, thank you for pointing this out!

Jochen Wilhelm . I carried on multiple regression analysis using all covariates as predictors to find out if any of these predict the performance on dependent variable. Results (Beta) indicated no significant results. Can I interpret my results saying these variables (analyzed as predictors in multiple regression) have so significant impact and thus proceed without using them as covariates or even conducting ANOCOVA.

P.S. same predictors are being analyzed after treatment. significant findings were noted among two of these predictors.

Be careful choosing how to include covariates, and be careful interpreting the results. Looking to see, for example, if the covariate has a big association with (or even effect on) the outcome variable does not necessarily mean you should include it. A common example (the birth weight paradox, I have a few paragraphs here and give more cites to research more focused on it, Article Estimating School Effectiveness with Student Growth Percenti...

Absence of evidence is not evidence of absence.