The effect size in statistics reflects the magnitude of a measured phenomenon. It may be expressed as a coefficient of determination (R2), which states how much variance can be explained by the controlled parameter.
If a direct and simple relationship between the variables is claimed (eg. concentration of a measurand vs. absorbance in determination of calibration plot for ELISA), the effect size may be close to 100%. Similarly, in pharmacology, the effect size of relationship between direct phenomena may significantly exceed 50%.
In social sciences, however, where the links between the observed phenomena are not so simple and largely multifactorial, I suspect, the effect sizes are much smaller.
How large the effect size (expressed as R2) in social sciences may be considered relatively "large"? Any examples? Any references?
Thank you for your help!
A lot depends on what you are measuring (or attempting to measure). If it is something as "soft" as attitudes, then you can expect a lot of measurement error, as well as a potentially large number of relevant omitted variables and interaction effects.
I did my graduate work in a very quantitatively oriented program, and we were taught that it was a favorable outcome when you could explain 30% of the variance in attitudes. Of course, that leaves three possibilities: your measures are highly unreliable (lots of measurement error), your model is mis-specified (you left out important predictors), or what you are trying to explain has a very large random component. Those are all serious problems, but when everyone in your field is struggling with those same problems, you learn to live with it.
A lot depends on what you are measuring (or attempting to measure). If it is something as "soft" as attitudes, then you can expect a lot of measurement error, as well as a potentially large number of relevant omitted variables and interaction effects.
I did my graduate work in a very quantitatively oriented program, and we were taught that it was a favorable outcome when you could explain 30% of the variance in attitudes. Of course, that leaves three possibilities: your measures are highly unreliable (lots of measurement error), your model is mis-specified (you left out important predictors), or what you are trying to explain has a very large random component. Those are all serious problems, but when everyone in your field is struggling with those same problems, you learn to live with it.
Use empirocal meta-benchmarks for most substantive interpetations.
Aguinis et al (2017).
Hope this helps,
Matt
In social science, Cohen gave the following reference level for effect size based on Pearson R:
-Small 0.10
-Medium 0.30
-Large 0.50
However, in real life, if a research can explain 50% of the data, it should be considered a failed project. Here, r = 0.50, for R-squared that is a mere 0.25 or 25%. Following Pareto 80/20, explanatory power should not be lower than 80%; any thing below that, it is a matter of insisting of being correct, i.e. Type I error.
The later work of Sawilowsky in restating Cohen's d as the measure
of effect size, the range seems more reasonable.
_______________________________________
Effect size d Reference
_______________________________________
Very small 0.01 Sawilowsky, 2009
Small 0.20 Cohen, 1988
Medium 0.50 Cohen, 1988
Large 0.80 Cohen, 1988
Very large 1.20 Sawilowsky, 2009
Huge 2.0 Sawilowsky, 2009
_______________________________________
See: https://en.wikipedia.org/wiki/Effect_size#cite_ref-Sawilowsky2009_16-0
LINKS:
(1) Cohen, Jacob (1988). Statistical Power Analysis for the Behavioral Sciences. Routledge. ISBN 1-134-74270-3.
(2) Cohen, J (1992). "A power primer". Psychological Bulletin. 112 (1): 155–159. doi:10.1037/0033-2909.112.1.155. PMID 19565683.
(3) Sawilowsky, S (2009). "New effect size rules of thumb". Journal of Modern Applied Statistical Methods. 8 (2): 467–474. http://digitalcommons.wayne.edu/jmasm/vol8/iss2/26/
- Badges
- Science topic
- Similar topics
- Psychology
- Psychometrics