(Smaller scores mean improvement, so negative value is beneficial)
Mean(SD)TxPre=19.2(4.1) Mean(SD)TxPost=15.4(2.8)
Mean(SD)ConPre=20.0(4.7), Mean(SD)ConPost=17.4(3.9)
n size both groups 14
My calculation is cohen d -1.08 for treatment, and -0.6 for control.
My conclusion would be a very large effect for treatment and 1.8x more effective than control (1.08/0.6).
Is there another way of calculating or reporting the overall effect of the treatment compared to control?
I've seen others calculate overall effect by comparing the means of the post-test group, using the change in each group and the SD of the post-tests. Is that appropriate? I calculated pooled SD of post-test at 3.39. In that scenario, the overall effect would be: (1.08-.6) / 3.39 = .14..... A trivial effect?
hi,
dear phil
Kelley, Ken; Preacher, Kristopher J. (2012). "On Effect Size". Psychological Methods 17 (2): 137–152. doi:10.1037/a0028086.
Ellis, Paul D. (2010). The Essential Guide to Effect Sizes: An Introduction to Statistical Power, Meta-Analysis and the Interpretation of Research Results. United Kingdom: Cambridge University Press
perhaps usefull
regards
Hi Phil,
I have used both of the techniques you describe above but have always included the 95% CI around the effect size point estimate. In your example, the point estimate may be 1.8x larger in the treatment group but without the 95% CI, we cannot be sure that there was true positive effect (i.e. the CI didn't cross zero or didnt overlap with the CI of the control group). In this way, the effect size and CI can also confirm the results of the traditional hypothesis test. I have included a paper that we recently did that used effect sizes and CIs in this way. I have also included an effect size calculator that we use but did not create. It is freely available on the web.
I hope this helps
Erik
Thanks Erik! I have used that calculator. love it! Agree about CIs. Would you consider a CI of effect size that includes zero to be "false" vs. a "true effect"? When we see a moderate effect size for example of 0.5, but CI includes zero, what do we conclude (assuming p>0.05 as well)? Appreciate your input. One more question: where is your interpretation of Hedges values from?
Hi Phil,
We have traditionally concluded that if a CI crosses 0 that there is no effect. But like everything else, it is likely on a continuum. A p value indicates the probability of that groups differ but we always default to a binary response at p=0.05. But if a 90% CI does not cross zero, then I am 90% confident that my treatment had an effect. If that is the case, then why should I completely dismiss my treatment just because I am not 95% confident. Thats the end of my rant :)
Interpretation of the values comes from Hedge's book Statistical Methods for Meta-Analysis.
- Badges
- Science topic
- Similar topics
- Medicine
- Physiology
- Biosignals
- Biomedical Signal Processing
- Statistical Analysis