A reviewer commented on my manuscript that the sample size in some of my treatment groups may not be sufficient to find a statistical difference (eg: n=10 in some age groups of animals, I used ANOVA in SPSS analysing the effects of age of animals on some biological measurements), so the reviewer suggested me to report data with residual error term or statistical power, instead of the standard error which I currently report with (data table and comments from the reviewer is attached)
Can anyone provide some guidance for me? Any example publications reporting data with residual error term or statistical power? Thanks in advance!
SPSS shows observed power in GLM menu (not by usual ANOVA menu option). Use General Linear Model-> Univariate menu option. Put your dependent variable into dependent box and your group variable into fixed factors box. Then from Options select Observed Power. This way you would get observed power in your ANOVA.
Residual error term (sum of squares) is also found in the same table in the row labeled "Error".
The important aspect in analysis is to analyse for the design that you used. In many questions it is not clear what the design was, you mention ANOVA - was it a survey or a science experiment ? Are you analyzing a factorial, are there blocks, and have you satisfied the assumptions of independent. Are your animals independent or is a cage or pen ? If you got a significant effect in ANOVA, and only then do you do use a mean comparison procedure to look at pairs of differences between means. Not sure if this helps? the other way to present the treatment means is the 95% Confidence intervals for a mean,
Observed power is essentially a monotonic transformation of the observed p value (in ANOVA). This means that you'd expect it to be > .5 if p < alpha (usually set at .05). This in turn means that it tells you absolutely nothing useful about the power of your study.
To go further you'd need to show your design had adequate power to detect the true effect or some reasonably estimate of it (given that if you knew the true effect you probably wouldn't do the study). Failing that adequate power to detect typical effects in your field. All of these estimates will be somewhat unreliable.
Is this a pre-registration or a manuscript reporting data already collected? If it is a pre-registration then it makes some sense for the reviewer to ask about power to detect a significant effect.
If you have already done it, then you either found a significant effect or not, so the probability is 0 or 1 (if you look in the box, the cat is either alive or dead). As Thom says, what some packages call "observed power" is just a disguised p-value so it seems odd to report. Can you provide a bit more context?
The table shows that your data is essentially completely inconclusive. Now the question is if there is something useful one can do with that study. Simply presenting it in its present form is worthless. However, *if* the study was designed to find relevant effects with a high probability, *then* your results can at least be used to state with resonable confidence that the effects are not relevant (that might not be the desired conclusion but at least it is a´conclusion). I think that this is what the reviewer wants to push.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics