Using a matlab package to calculate effect sizes (MEStoolbox), I chose to evaluate g_psi for the effect sizes between a control group and 12 treatments. In a previous boxplot of samples, I can identify three groups that differ extremely from the rest, but the anova results in significant differences between all treatments since the sample size is so large. That is why, I want to use Hedges' g to visualize that three treatments differ more than the others.
1. Question: What is more meaningful, an evaluation of the reference with each of the treatments and collecting results of Hedges' g or one evaluation with all treatments together?
2. Question: Do you know any comprehensive visualization of effect sizes that allows an easy interpretation?
Why not just showing the boxplot? It visualized all this information quite well.
Done already, the boxplot is qualitatively fine, although my confidence intervals are not larger than the point itself. (Sample size 8500) Since I need to prove that some distributions differ more than others, a classification of effect sizes is useful. What do you think about using the range of 0.2-0.5 for small, 0.5-0.8 for medium and > 0.8 for large effects?
I don't get the point... You *do* show that the means are different, and you *do* show that the precision of theses estimates is far better than the differences between the means. What else do you want? If the value for group B is 5 +/- 0.000001 and for the group C is 6 +/- 0.000001, who do you think would not believe that there is a difference between B and C? Or if you have D with 7 +/- 0.000001, then it is also obvious that (D-B) > (C-B). The confidence interval for this "difference of differences may be obtained by error propagation from the CIs of the means. Since these intervals are so tiny, the propagated CI will still be tiny...
Question 1: It depends! You can compare the (mean) effect size of one treatment over all treatments (a) or all treatments over the control (b) or one treatment to the control. (a) "Treatment A had a significantly higher mean (patients survival, foxes population survival...) than other treatments." (b) "Only three treatments A, B and C had a significantly higher mean (patients survival, foxes population survival...) than control." (c) "Treatment A had a significantly higher mean (patients survival, foxes population survival...) than control."
Combine these as you like in the context of your study. Imaginary example, you make the tests b and c, one could say: "From all methods of habitat restoration only three had a significantly higher foxes mean survival rate than agricultural landscape (control). However, the method of restoration A was the most efficient in term increase of foxes survival rate."
Question 2: The easy way is incorporating results from similar study. If it is not possible, here is Cohen's rule to interpret effect size. Below the effect is small at or below, "medium" around 0.5 and important at and above 0.8. However, it is situation dependent and you are in the best position to decide if 0.5 is enough. If your g is 0.5 but only one more fox survived with method A and other treatment had similar survival rates... I would say that one fox out of 30 for example might not be enough to call it a day.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA