Over the past couple of years, I have been getting into methods to infer causality (after a strong focus on correlation, survey-based research). Methods such as longitudinal analysis using fixed effects as well as DiD-analysis are contested. But are there other valid methods to infer causality apart from actual experiments or the above-mentioned methods? And what are the benefits and costs of such methods? Looking forward to hearing what you think.
Some of the best work in this area was done by David Freedman of the University of California. I suggest you start with an excellent paper of his that is attached. Best wishes, David Booth
Good references include (the Morgan and Winship has a newer edition, and is probably the most introductory):
Pearl, J. (2009), Causality: Models, Reasoning, and Inference (2nd ed.), New York: Cambridge University Press.
Pearl, J., Glymour, M., and Jewell, N. P. (2016), Causal Inference in Statistics: A Primer, Chichester, UK: Wiley.
Elwart, F. (2013), “Causal Graphical Models,” in Handbook of Causal Analysis for Social Research, ed. Stephen L. Morgan, New York: Sage Publications, pp. 245–273.
Imbens, G. W., and Rubin, D. B. (2015), Causal Inference for Statistics, Social, and Biomedical Sciences: An Introduction, New York: Cambridge University Press.
Morgan, S. L., and Winship, C. (2007), Counterfactuals and Causal Inference: Methods and Principles for Social Research, Cambridge: Cambridge University Press.
The question seems to suggest that making causal inference is simple with random allocation. Assumptions are still needed.
And, perhaps "The Book of Why: The New Science of Cause and Effect" (Pearl & Mackenzie, 2018) should be added.
John
Cook, Shadish & Wong had an interesting article I remember reading - I'm sure there's more current thinking on this, but this one stood out to me.
Article Three Conditions under Which Experiments and Observational S...
Also a really fascinating history of the use of regression discontinuity in social science research by Thomas Cook.
https://www.sciencedirect.com/science/article/abs/pii/S0304407607001108
“Waiting for Life to Arrive”: A history of the regression-discontinuity design in Psychology, Statistics and Economics
This paper reviews the history of the regression discontinuity design in three academic disciplines. It describes the design's birth and subsequent demise in Psychology even though most problems with it had been solved there. It further describes the scant interest shown in the design by scholars formally trained in Statistics, and the design's poor reception in Economics from 1972 until about 1995, when its profile and acceptance changed. Reasons are given for this checkered history that is characterized as waiting for life to arrive.
Daniel Wright - which assumptions do you mean tied to randomization? Things like balance tests, manipulation and attention checks?
Bert George , yes, those, and more. Everything but the ceteris paribus part. One I often struggle with is whether the manipulation is the hypothesized cause (the do(x) in Pearl's terminology) and only that. Often people test if the manipulations changes what it is supposed to change, but not test to make sure it doesn't affect other things (and this is tough, how do you test that for it not affecting everything else, and then how do accept the null if appropriate). I give an example on p. 203 of Article CAUSAL AND ASSOCIATIVE HYPOTHESES IN PSYCHOLOGY: Examples Fr...
where the authors did a manipulation to change stress levels, but it would have also prompted a myriad of different emotions.Random allocation is a very useful tool, particularly in program evaluation (where the program implementation is the cause). Some very bright people in the history of science/math/stats have struggled with the notion of causality.
Thanks Daniel Wright - I am particularly intrigued by your reference to program evaluation. I do a lot of policy and strategy work in gov and am seeing more and more behavioral insights teams applying randomization as a strategy tool to experiment and balance different strategic choices. Also thanks Sebastian Jilke
- I do have a copy of your book with James and Van Ryzin and you are right to promote it. It’s a great read for PA scholars interested in experiments.
A good source for program evaluation is https://www.povertyactionlab.org/ . It seems randomization has become more popular among economists recently.
If you're researching entrepreneurship and innovation-related policy, I highly recommend the Innovation Growth Lab ( https://innovationgrowthlab.org/ ). They have excellent resources for policymakers to conduct experiments (mostly related to innovation/entrepreneurship, but can easily be extrapolated to other fields). They do great work in engaging with policymakers to introduce the idea of policy experimentation.
They also have some excellent grants programs for researchers (full disclosure: they are funding some of my ongoing work). Very well designed application process with strong developmental feedback in the design stage.
Qualitative methods and case study research are by far the strongest ways to study causality
- synthetic control methods (amazing, when they work- check it out on Abadie & Heinmuller page).
- instrumental variables (hard to do right)
- natural experiments (easy to do when you are lucky with your dataset, but otherwise very hard to come by)
Cheers!
Definitely agree with Simone. As well as looking for causation in a laboratory or spreadsheets why not try ethnography and observation? These techniques really do yield longitudinal data whereas a lot of statistical methods that are described as longitudinal rely on a series of cross sectional measures.
- Badges
- Science topic
- Similar topics
- Computer Communications (Networks)
- Simulators