What are the factors that need to be considered when determining the sample size in research?
Hello Hameed,
This is a topic that shows up frequently, both on ResearchGate and a host of other forums.
A. If the purpose of the study is to estimate a population parameter, then the relevant factors include:
1. The target precision with which the parameter is to be estimated;
(Better precision = larger sample size)
2. The target risk that the resultant CI estimate will completely miss the true value;
(Lower risk = larger sample size)
3. The amount of variance for the attribute of interest in the population;
(More variance = larger sample size)
4. The specific probability sampling method used;
(For example, stratified random sampling can be more efficient than simple random sampling)
5. Population size.
(The larger the population, the less of a difference population size makes in the derivation of a target sample size.)
B. If the purpose of the study is to test a hypothesis, then the relevant factors include:
1. The smallest degree of difference or relationship that you'd like the study to be capable of detecting, should it actually exist (the target effect size);
(Smaller ES = larger sample size)
2. The target statistical power (probability of correctly rejecting a false null hypothesis);
(Higher power = larger sample size)
3. The target type I error risk (alpha level);
(Smaller risk = larger sample size)
4. The specific statistical test/statistical hypothesis that would be applied;
(Varies by test)
5. For multi-variable studies, the number of IVs and DVs.
(Generally, more DVs = larger sample size)
Of course, for either scenario, the very real concerns of cost and logistics may result in less than statistically optimal sample sizes having to be used.
An excellent resource for (A) parameter estimation (sampling) studies is the text by W. G. Cochran (1977). Sampling techniques (3rd ed.). New York: Wiley & Sons.
An excellent resource for (B) hypothesis testing studies is the text by J. Cohen (1988). Statistical power analysis for the behavioral sciences (2nd ed.). New York: Routledge.
Good luck with your work.
Hi David Morse , I am interested if you could expand on point 5, since that raises an issue I am often asked. It reminded me of this discussion between Cochran and Fisher (from Rosenbaum OS, but more accessible in http://www-stat.wharton.upenn.edu/~rosenbap/CochranCausalCrossword.pdf).
"About 20 year age, when asked in a meeting what can be done in observational studies to clarify the step from association to causation, Sir Ronald Fisher replied: 'Make your theories elaborate.' The reply puzzled me at first, since by Occam's razor, the advice usually given is to make theories as simple as is consistent with data. What Sir Ronald meant, as subsequent discussion showed, was that when constructing a causal hypothesis one should envisage as many \emph{different} consequences of its truth as possible, and plan observational studies to discover whether each of these consequences is found to hold."
From this, it seems having many DVs is good, but the power for detecting all of them is lower that detecting show a random one (or a most likely to yield an effect one). Even if you had a fixed sample size, would you suggest to a student to measure more DVs? Would you suggesting detecting some, changing alpha, beta, ES, or something else?
Hi Daniel Wright ,
My personal belief is that life is truly multivariate, and the classic experimental paradigm of "isolating" a single IV to observe its influence on a single DV has little relationship to the real world.
In general, I would seldom, if ever, advise someone to jettison variables pertinent to their backing theory and/or research question/s. However, I would want them to be aware of the consequent features of any statistical analysis planned as it was affected by the number of variables (esp. DVs). Forewarned is forearmed.
In such cases, compromise judgments between type I and type II risk levels, target ES, and whatever resource limitations might exist vis-a-vis possible sample size, may well be called for. As was once suggested, "...surely God loves the .06 [type I error risk] level nearly as much as the .05." As well, do note that the usual reliance on the .05 level appears as much to have come from Fisher's classic text, Design of Experiments. (See this thread for the folks that brought that to my attention: https://www.reddit.com/r/statistics/comments/kg16cp/d_q_surely_god_loves_the_06_nearly_as_much_as_the/)
Sorry, didn't mean to steer this thread sideways!
- Badges
- Science topic
- Similar topics
- Correction