The question is, What is the logic of case selection in comparative cases study research? Because of the theoretical issues to be explored and tested, the small-n comparative case study is the appropriate approach to research (Lijphart, 1971; 1975). Keeping in mind the benefits, in terms of internal validity, that experimentation offers and the confidence in causal inferences that it provides, the proposed research strategy optimizes control and effectively isolates the relationships of interest, given the constraints created by our need to observe the phenomenon contextually. One should try to articulate such a method by relying on a logic of case selection that, within the limits inherent in the well-designed small-n comparative case-study (Verba, 1967; Eckstein, 1975; Yin, 1984), allows the researcher to maximize the internal and external validity possible given his/her contextual interests, thus increasing the confidence and generalizability of our causal explanations. Careful attention to the issues of case selection, in case study research, is a critical component of defensibility. The challenge is to determine the research strategy and case selection method most appropriate to the investigators’ theoretical concerns and to their desire to make confident inferences from the findings. How should we do it?

The answer is purposeful sampling of cases for comparison from the universe of cases. Purposeful sampling enables researchers to move away from the indeterminacy that makes generalizations from case studies problematic and in the direction of valid causal explanations of social and political phenomena; for example, the role that research-based knowledge has played in preparing for a probable disaster. Where these concerns pertain, selection of cases in small-n comparative case study research should be guided by some theoretically-driven decision rule. Such a selection process is referred to as theoretical sampling (Mitchell, 1983; Scott, 1987: 158). Cases selected in this manner vary on identified characteristics of theoretical interest. As Fernandez 2005) notes, if researchers hope to explain variation in a dependent variable, the choice of cases must allow for variation in the dependent variable. King and his co-authors (1994, 129) put it bluntly when they ask, “How can we explain variations on a dependent variable if it does not vary?” (p. 129).

I (Goggin, 1986) have identified one common problem that plagues many who design and carry out comparative policy research, the “too few cases/too many variables problem”. One remedy for the too few cases/too many variables problem is to select cases with an eye toward maximizing similarities among cases except for the phenomenon to be explained or maximizing differences among cases except for the phenomenon to be explained (Goggin, 1986:333-34). Przeworski and Tuene (1970:39) label these "most similar" and "most different" systems designs, respectively. The authors describe the differences in these two logics of case selection as follows: “The most similar systems design is based on a belief that a number of theoretically significant differences will be found among similar systems and that these differences can be used in explanation. The alternative design, which seeks maximal heterogeneity in the sample of systems, is based on a belief that in spite of intersystem differentiation, the population will differ with regard to only a limited number of variables or relationships.” The logic of the two designs is identical, with co-variation or the lack thereof, as the instrument for distinguishing relevant from irrelevant variables. It is only the incidence of variability in the dependent variable – some variability in the most similar systems approach and no variability in the most dissimilar systems design – that differentiates these two approaches to case selection in comparative research.

The question is, What is the logic of case selection in comparative cases study research? Because of the theoretical issues to be explored and tested, the small-n comparative case study is the appropriate approach to research (Lijphart, 1971; 1975). Keeping in mind the benefits, in terms of internal validity, that experimentation offers and the confidence in causal inferences that it provides, the proposed research strategy optimizes control and effectively isolates the relationships of interest, given the constraints created by our need to observe the phenomenon contextually. One should try to articulate such a method by relying on a logic of case selection that, within the limits inherent in the well-designed small-n comparative case-study (Verba, 1967; Eckstein, 1975; Yin, 1984), allows the researcher to maximize the internal and external validity possible given his/her contextual interests, thus increasing the confidence and generalizability of our causal explanations. Careful attention to the issues of case selection, in case study research, is a critical component of defensibility. The challenge is to determine the research strategy and case selection method most appropriate to the investigators’ theoretical concerns and to their desire to make confident inferences from the findings. How should we do it?

The answer is purposeful sampling of cases for comparison from the universe of cases. Purposeful sampling enables researchers to move away from the indeterminacy that makes generalizations from case studies problematic and in the direction of valid causal explanations of social and political phenomena; for example, the role that research-based knowledge has played in preparing for a probable disaster. Where these concerns pertain, selection of cases in small-n comparative case study research should be guided by some theoretically-driven decision rule. Such a selection process is referred to as theoretical sampling (Mitchell, 1983; Scott, 1987: 158). Cases selected in this manner vary on identified characteristics of theoretical interest. As Fernandez 2005) notes, if researchers hope to explain variation in a dependent variable, the choice of cases must allow for variation in the dependent variable. King and his co-authors (1994, 129) put it bluntly when they ask, “How can we explain variations on a dependent variable if it does not vary?” (p. 129).

I (Goggin, 1986) have identified one common problem that plagues many who design and carry out comparative policy research, the “too few cases/too many variables problem”. One remedy for the too few cases/too many variables problem is to select cases with an eye toward maximizing similarities among cases except for the phenomenon to be explained or maximizing differences among cases except for the phenomenon to be explained (Goggin, 1986:333-34). Przeworski and Tuene (1970:39) label these "most similar" and "most different" systems designs, respectively. The authors describe the differences in these two logics of case selection as follows: “The most similar systems design is based on a belief that a number of theoretically significant differences will be found among similar systems and that these differences can be used in explanation. The alternative design, which seeks maximal heterogeneity in the sample of systems, is based on a belief that in spite of intersystem differentiation, the population will differ with regard to only a limited number of variables or relationships.” The logic of the two designs is identical, with co-variation or the lack thereof, as the instrument for distinguishing relevant from irrelevant variables. It is only the incidence of variability in the dependent variable – some variability in the most similar systems approach and no variability in the most dissimilar systems design – that differentiates these two approaches to case selection in comparative research.

comparative research:

1. Enough sample size

2. Match samples: compare apple to apple NOT apple to orange

3. Randomization

4. Selecting independent and covariates carefully 

Methods may differ depending on the SUBJECT AND SCALE of comparative research. In comparative research about  legal and public policy issues,  entire nations or sub-jurisdictions may be the "population" of the research.  Randomization is neither justified nor practical.  the issue is identifying "explanatory variables" is much more complex and in some ways, untenable (too many unknowns, intervening and confounding variables).  However, the usefulness of comparative research of legal and institutional-policy issues is by no means lesser.  See my paper: Comparative Research at the Frontier of Planning Law.  http://www.lawlectures.co.uk/w113/documents/ijlbe-editorial-alterman.pdf

We have just conducted a study of how to assess the cross-national transferability of policy measures which you might find useful. This provides a comprehensive review of the issues involved in determining the transferability of policies from one country to another. I attach it. It is also available on Research Gate. 

Technical Report Assessing the cross-national transferability of policy measu...

My colleagues and I at the University of Colorado Denver School of Public Affairs are also working on what is closely related to "transferability of policy measures," namely, the diffusion of policy innovations from one level of government to another, from one city to another, and from one country to another. The policy domain is disaster management. We are planning a book that compares the uses of research-based knowledge across the U.S., Canada, and several European countries and are planning to participate in the COST meetings in Europe next fall.

Malcolm

It would be interesting to know whether you come to similar conclusions as us about how to evaluate the transferability of policy measures, albeit in a different domain/field of inquiry. I look forward to hearing from you. If you ahve written anything, it would be good to see it.

Colin

Colin,

We have written several papers based on comparative case studies of U.S. cities' acquisition and use of research based knowledge in planning for and adaptijng to disasters due, at least in part, to climate change, but we are only beginning to develop ideas about the diffusion of knowledge, both from the ground up and the top down. One example is New York City, which in the aftermath of Superstron Sandy turned to Amersterdam for ideas about how to prepare for the next Sandy. Another example is that some cities in our sample turned to the cities of New York and Balrimore for innovative ideas about how best to deal with low probability/high cost events. After I have a chance to read your latest paper I will see if anything that we have already written might be directly or indirectly related.

Malcolm

A qualitative principle for a comparative study would be haveing at least of common point, or a sense of some sort of closeness with the subject,to begin with; just a collective data of something to compare would not be an enlighting avenue on finding the creative transparencies; then, the rest it (right number of hypothesis, and non- bias evaluations) would flow smoothly.

Thanks so much for your inputs, I'm really learning a lot

Lijphart's work is seminal and I appreciate Goggin's and Williams's comments. I think there is another dimension that has not been mentioned: comparison of scale. That is, comparing a national issue to the same issue in a smaller state or province. For example, is it possible/useful to compare sex education in schools related to out-of-wedlock birthrates between a country and a state? Many variables (too many?) may emerge and cultural context must be considered.