I recently did an Analysis of Covariance (ANCOVA) where some test scores were adjusted for a covariate. The post-adjusted scores were about 2 points lower than the unadjusted scores. The pre-adjusted scores had a standard deviation of around 10, and the post-adjusted scores had a standard error of 1.9. The effect of adjustment was not statistically significant.
If I were to report this result, would I say that the effect of the covariate on the scores (2 points) was well within one standard deviation of the unadjusted scores, or instead say it was slightly more than the standard error of the adjusted scores? If the latter is the case, then could I say that another study with a slightly larger sample size might make the effect of adjustment become statistically significant? Any experience and rationale regarding this would be helpful... thanks
Given the question that you are trying to answer, I think you might find it easier to interpret this analysis if you did it as a regression, rather than an ANCOVA (they are algebraically equivalent, but they report the results in different fashions).
To do such a regression, you simply enter both the covariate and treatment variables as independent variables in the regression, where the treatment variable should be scored 0-1 (if you have more than one condition in the treatment, then you need to enter a set of "dummy variables").
The results will give you direct tests for the significance of both the treatment and the covariate.
Thank you David, that helps somewhat, but my conundrum lies with the interpretation of the results. Two points is greater than the standard error of the adjusted scores; but it is a trivial fraction of the standard deviation of the unadjusted scores. Would this not suggest that if the experiment were to be repeated (on different populations for example) that the covariate's influence would be enough to significantly change the (adjusted) sample mean? I 'm unsure which measure of variation (the unadjusted standard deviation, or the adjusted standard error) I should relate this two-point difference to.
The difference between standard error and standard deviation is that the first takes the sample size into account, which is why larger samples have greater "power" (i.e., they have smaller standard errors). For statistical purposes, based on sampling theory, you use the standard error to assess statistical significance.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size