For medical researchers, "external validity" seems to show up as a limitation in RCTs. I think medical researchers typically deal with the issue of generalizability -- external validity -- through meta-analysis. Economists can sometimes run several RCTs and complete a meta-analysis (see, for example, the MIT poverty lab), but in the case of large field experiments, the number of times a similar study can be run is limited. So you end up with one or two field experiments on a given topic, and concerns about external validity.
Deaton and Cartwright address this issue in the context of economics. However, it often seems to me that philosophers argue that results of a single trial either "are" or "are not" externally valid, while economists look a bit more closely at the context to which the results are about to be generalized. When they do that, they are (presumably) importing knowledge from other research or from history to determine whether the new application is sufficiently similar to the experimental situation to allow generalizability. Is this a substitute for a meta-analysis?
I think the issue of external validity or generalizability should not be restricted to randomized control trials alone, it should be applicable to most research designs, cross-sectional, case-control, cohort and others.
External validity should depend on quality of the research, the setting, methodology, limitations etc.
- Badges
- Science topic
- Similar topics
- Biological Science
- Kinesiology
- Running