Hi! I've been trying to perform a power analysis for a one-way ANOVA in R using the pwr package. The equation that is employed for this is based on Cohen's f and assumes that variances among the treatments are exactly the same. In fact, as can be seen in the equation from this web page (https://www.statmethods.net/stats/power.html), we have to introduce a single value for the variance. Nevertheless, in my case, despite variances are statistically equivalent (based on the Levene's test) I do not know which variance value (of the treatments) I should introduce in the equation. The examples I have seen so far consider hypothetical cases where all treatments have the same exact variance: e.g. 8.5.
How should I proceed?
"despite variances are statistically equivalent (based on the Levene's test)" -- be careful with your arguments here. A non.significant test result is no indication of equivalence or no difference. "Absence of evidence is NOT evidence of absence" (J.Box)!
Small differences are not a problem. If you think that the homogeneity of variance
assumption is problematic, you should go and rethink the model and the statistical question you ask. It might make not much sense in the first place.
If you need that for a power analysis, you need an educated guess about what variance you would reasonably expect. You can get an estimate from the data at hand, and then the pooled variance estimate may be the "best". However, keep in mind that the point estimate may be quite uncertain. Look at the confidence interval of this estimate, and check if assuming a variance a bit closer to the upper limit may be more secure (something you can also easily adjust by choosing a higher power, but this is sometimes harder to argue in front of the person responsible for the budget and particularily in front of an ethics committee).
Depends for what you need this power calculation? Sample size calculation for a trial? Then you should assume a variance you might expect at maximum in the trial "as worst case" and you could 95%CI as orientation as Jochen suggested. And it depends how "conservative" you want to be? Because of these "awkward" effect sizes by Cohen in the -pwr- package I usually just use the power.anova.test() function where I can give the within- and between-variance seperately. If you want to use the -pwr-package than use either the pooled within-variance or be conservative and use the maximum within-variance for calculating f.
A total different way would be just using Monte-Carlo simulation.
Hi, thank you for your advice and reply, particularly when using Levene's test as Jochen mentioned. I think the largest within variance will be useful now since this is a trial to start the next step of experimentation. Thank you Stefan, it is definitely better to use conservatives values now.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Applied Statistics