We are about to publish research into the effectiveness of a (commercial), non-mental health growth training for normal adults. Its goals are things such as personal effectiveness, self-confidence and assertiveness. Several searches of the literature did not yield any other such effectiveness study. (We're extremely interested in anyone who could point one out.) This leaves us with a profound lack of findings with which to compare ours. Any empirically based frame of reference seems to be lacking.
We have computed Cohens d's which are 'off the charts'. However, Lipsey rightfully warns against a general interpretation of d-values, saying: "(...) The widespread indiscriminate use of Cohen’s generic small, medium, and large effect size values to characterize effect sizes in domains to which his normative values do not apply is (thus likewise) inappropriate and misleading (…)”.
Interventions around drug-dependency may be comparatively effective when small, while those aimed at for instance stress reduction might be comparatively small while typified as large.
We imagine growth trainings to be comparatively effective since people enter them voluntarily and are mostly highly motivated to change.
http://ies.ed.gov/ncser/pubs/20133000/
A nice way of interpreting Cohen's d is to look at the percentile standing of the new mean compared to the population before treatment. For example, d = 1 means that after treatment, the new mean is in the 84th percentile of the variable in question before treatment. Here's a table that gives simple conversions:
http://www.uccs.edu/lbecker/effect-size.html
Thanks for this (and highlighting the relevant box). I'll look into it and will see if I can use it, although I was forced by circumstance to use a pre-post-follow-up design because constructing a control group was plainly impossible.
I guess you realize my question was quite a different one.
I cannot point out a single study from this context, as you requested, but it is may be helpful noting, that effect sizes, especially cohens d for repeated measurements actually reflect statistical needs for instance when an experimental power analysis is done, as well as describing results in reference to a base population as David Dunaetz put it out.
In a repeated measurement d can be calculated in two ways. The first way is the standardized difference of the growth changes from zero. This one is used for power analysis, and will work for your purpose, if you are simply trying to find out which intervention of two is more effective. The second possibility is to incluce random varience by calculating the standardized difference between the means of both repeated measurments as groups. This one can be interpreted in terms of a "success" rate, which David also put out. Thus, in terms of "How many percent of the people feeling better after an intervention on a "real world" scale?" it may be well an indicator of ecological effectiveness.
Hope this helps a bit. Cheers
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA