If a moderate effect size is calculated when the p significance value is greater than 0.05, what does this indicate and how should it be reported?
It could mean that, even though your observed effect was sizeable, your study lacked power (e.g., due to a small sample) to reject the null hypothesis of a zero effect.
I would recommend reporting the results as is. That means reporting the effect size along with the p value for the null hypothesis significance test; or, even better, report the effect size together with a confidence interval (CI) that corresponds to your chosen level of alpha (e.g., a 95% CI for alpha = .05).
To help answer your question, you also need to think about the implications of P-values with respect to reproducibility and rejecting the the null hypothesis and you shouldn't have (AKA type I errors). I am attaching an image from Jim Frost's book called "Hypothesis Testing" (pg.99 of 394) where he shows that high P-value studies are associated with less reproducibility rates (inspired by the following paper: Article Estimating the reproducibility of psychological science
).As for effect size, I will leave this sentence "You can have a large effect size, but if your sample size is small and/or the variability in your sample is high, random error can produce substantial differences between the groups" (Jim Frost's book called "Hypothesis Testing" (pg.103 of 394). I would highly encourage you to read Chapter 4 - "Interpreting P-values" of this book for this topic of P-values, effect size and how it all fits together. It is a great and easy read!
- Badges
- Science topic
- Similar topics
- Medicine
- Counseling Psychology