In my study I was looking at plasma levels of a compound over time in a total of 40 volunteers that harbor 3 different genotypes. The stats used were 2-way ANOVA repeated measures, time (10 time points) was the within subject factor and genotype (3 genotypes) was the between subject factor. Can anybody help please?
Why do you want to calculate post-hoc power? If it's to plan a follow-up study that's legitimate. If it's to help you interpret your data it's not useful at all. The post-hoc power you calculate is simply a one-to-one function of your p-value and, as such, it does not tell you anything that you're not getting from the p-value. In particular, it cannot distinguish between "I failed to reject but my power was low so maybe the null hypothesis really is false" and "I failed to reject but my power was high so maybe the null hypothesis is really true". If your p-value was high your calculated post-hoc power has to be low and if your p-value is low your calculated post-hoc power will come out high. This was discussed in The American Statistician by Hoenig and Heisey (2001): The abuse of power: the pervasive fallacy of power calculations for data analysis. American Statistician 55(1):19-24.
easiest thing to do might be download G*Power 3 from http://www.gpower.hhu.de/en.html. It's free and quite detailed.
Why do you want to calculate post-hoc power? If it's to plan a follow-up study that's legitimate. If it's to help you interpret your data it's not useful at all. The post-hoc power you calculate is simply a one-to-one function of your p-value and, as such, it does not tell you anything that you're not getting from the p-value. In particular, it cannot distinguish between "I failed to reject but my power was low so maybe the null hypothesis really is false" and "I failed to reject but my power was high so maybe the null hypothesis is really true". If your p-value was high your calculated post-hoc power has to be low and if your p-value is low your calculated post-hoc power will come out high. This was discussed in The American Statistician by Hoenig and Heisey (2001): The abuse of power: the pervasive fallacy of power calculations for data analysis. American Statistician 55(1):19-24.
The confidence intervals will be a good clue to whether you are truly underpowered.
The confidence interval will tell you how well you've done. It contains all values of the parameter that cannot be refuted by the data. If you're satisfied with the narrowness of the confidence interval, then you've done a "good job". If you think the confidence intervals are too wide (you haven't narrowed down the possibilities enough) then you, sadly, should have had a bigger sample size.
Thanks all for your answers! I am actually using Gpower calculator to look into this but I was not sure which values from my data should I use for the "effect size" parameter...I had so many means and std from each time point from each group that I wasn't sure how to get just only one...
My intention calculating the power was never to interpret my results but just give me an idea of how powerful was my study, I know that the N number would be a limitation. Thanks both for your points and clarification, I will definitely calculate the confidence intervals instead. Really appreciated your help!
In addition to the above comments, the CI are perhaps the easiest to interpret. There are more than one ways to define effect size. If you want to compare just two populations at a single time point you could use percent change from baseline or Cohen's D (which I like because it builds in a signal-to-noise measure). For ANOVA and repeated measures ANOVA an effect size is usually defined from some ratio of sums of squares but it is harder to interpret because it is the effect size in a global sense (in your case over 3 genotypes and across 10 time points). My point of reference on that is Bakeman, Recommended effect size statistics for repeated measures designs, Behavior Research Methods 2005, 37(3), 379-384.
There is also Russ Lenth's power calculator online that has a rather intuitive interface.
The power of a test should be for the minimum effect you want to detect. Using a previously found effect size in a power calculation implies that this is the minimum effect size that you want to design your study to detect. My question is why would this be?
The previous observed effect size might inform what range of effect size you think you will likely observe, but it will not tell you the effect size you want to design your study to detect. What may be useful are things like the previously observed standard deviation. If you know the minimum effect size in raw units, then the previous SD will help you to estimate what might be the minimum standardized effect size which is what software like G*Power like.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA