I am interesting in calculating sample size for a descriptive design, quantitative research method study. I will use a nonprobability (purposive) sampling technique. Realizing that a probability statistical technique cannot be used to calculate sample size; it is not statistically justified to use probability statistical measurements (e.g. confidence level) and sample size because the probability of participants' opportunity to participate is unknown. Looking for calculation method and/or literature, studies that may provide insight. Thank you.
NON-PROBABILITY SAMPLING: This is equally acceptable method of obtaining data when the followings are observed: (i) no sampling frame; (ii) bias is negligible; (iii) sampling and sample could be monitored (chain of control and custody: who, what, when, why, and how?); (iv) probability sampling is not feasible or appropriate, i.e. if the data is time dependent and the there is a significant difference between the base period and the current period as result of time lapse (economic data); (v) data could be collected only where the collectors are available; (vi) by the circumstances and nature the sample size is small and limited, i.e. forensic evidence cases; and (vii) sampling decision is made in the field or at an operational level, i.e. QC managers select widgits from a production batch to do a "pre-test" for defects before invest resources for the "big test."
HOW TO DO NON-PROBABILITY SAMPLING?: There may be many methods used. However, these two methods may be instructive: (i) quota sampling, and (ii) representative item sampling.
(i) Quota Sampling: Given a population with non-finite size (N unknown), the researcher define all observable characteristics of the population. Determine the ration of such characteristic in the population through series of test samples. Once this proportion is known, start taking sample for the study for a "size" where the proportion of the pre-defined characteristics comes close. Here, percentage confidence becomes applicable.
(ii) Representative Item method: The researcher is looking for a particular characteristic or a set of properties (this is the purpose). The population may have characteristics: [a, b, ..., z], but the research is searching for only characters (d, e, f). First take series of sample to get an average (expected) proportion of (d,e,f) in the population, then take the "sample" needed for the study by taking the sample until the same proportion shows. This is similar to quota sampling, but in representative method, one takes look for specific characteristics: (e,d,f).
n = (Z2σ2) / SE2
... where n = minimum sample size; Z = standard score obtained by (Xi - X^) / S; σ = estimated population standard deviation; and SE = standard error where SE = σ / sqrt(n). Note that Z provides a critical value for which F(Z) = % confidence may be read from the Z-table. The fact that the sample is non-probability does not preclude us from calculation the probability among the test samples that we took. Except there is one caveat, do bootstrapping to verify if the n obtained from one trial to the next varies significantly. It is not true to say that non-probability sample precludes probability analysis.
BIAS: The concern in non-probability sampling is "bias." since the sampling is "purposive"---i.e. taking the sample for a particular purpose, we are concerned that the result may be self-fulfilling prophecy. So long as this concern may be overcome, bias minimized, then even non-probability sampling is acceptable----acceptability enough to send a person to the execution chair in case of using limited and very small genetic evidence in prosecuting a criminal in a murder case. Therefore, if non-probability sample is used, the researcher must be able to demonstrate safeguarding measures against bias. In forensic evidence, this is generally accomplished through the chain of custody.
RELIABILITY: Reliability is defined as the ability to reproduce the same result in subsequent studies following the same procedures used in the prior study. If the result could not be reproduced, the prior study fails on reliability test. Whatever sample method used, if subsequent independent study could reproduce the result it is said that the study is reliability. In genetic test, researchers are provided with limited amount of sample, generally small amount. Regardless of how the genetic material was taken, probability or non-probability sampling, if subsequent test does not reproduce the same result, the conclusion of the prior study is not reliable and, therefore, unreliable.
SAFEGUARD BY DISTRIBUTION TEST: As a precaution use distribution verification to verify if the population is normally distributed. If it is normally distributed, one may safely say that the elements in the population have equal chance of being selected. If all elements in the population exists in a state of random (equal chance of being selected), no matter if we "conveniently" or "randomly" select the elements, we should get the same result. Distribution test may be accomplished by: (i) Shapiro-Wilk Test, (ii) Cramer-von Mises Test, or (iii) Anderson-Darling Test. if the test shows that the population is normally distributed, then all elements in the set has equal chance. If the distribution test shows that the data are not normally distributed, use the chi-square test as back up for your significance test.
VALIDITY: Validity refers to the precision between the expected and the observed. in the genetic testing example, since the genetic make up of everyone is different, the result of the test----even with minute sample---must hit the mark in every test. If one test identifies the owner of the gene to be Mr. A and on subsequent test shows that the gene comes from Mr. B, the test is not valid---here, the choice of sampling method (probability or otherwise) becomes moot.
REFERENCES:
[1] Laplace, Pierre-Simon de (1774). “Mémoire sur la probabilité des causes par les événements.” Mémoires de l'Académie royale des Sciences de Paris (Savants étrangers), tome 6: 621–656.Translated by Stephen M. Stigler in Statistical Science 1 (3), 1986
[2] Lexis, Wilhelm (1878). “Sur la durée normale de la vie humaine et sur la théorie de la stabilité des rapports statistiques.” Annales de démographie internationale (Paris) II: 447–462.
[3] Lukacs, Eugene; King, Edgar P. (1954). “A Property of Normal Distribution.” The Annals of Mathematical Statistics 25 (2): 389–394.
[4] Marsaglia, George; Tsang, Wai Wan (2000). “The Ziggurat Method for Generating Random Variables.” Journal of Statistical Software 5 (8).
[5] Marsaglia, George (2004). “Evaluating the Normal Distribution.” Journal of Statistical Software 11 (4).
[6] Papoulis, A. Probability, Random Variables, and Stochastic Processes, 2nd ed. New York: McGraw-Hill, pp. 100-101, 1984.
[7] Rohrbasser, Jean-Marc; Véron, Jacques (2003). “Wilhelm Lexis: The Normal Length of Life as an Expression of the “Nature of Things.”” Population 58 (3): 303–322.
[8] Stigler, Stephen M. (1982). “A Modest Proposal: A New Standard for the Normal.” The American Statistician 36 (2): 137–138.
[9] Wallace, C. S. (1996). “Fast pseudo-random generators for normal and exponential variates.” ACM Transactions on Mathematical Software 22 (1): 119–127.
[10] Whittaker, E. T. and Robinson, G. “Normal Frequency Distribution.” Ch. 8 in The Calculus of Observations: A Treatise on Numerical Mathematics, 4th ed. New York: Dover, pp. 164-208, 1967.
What is your goal? Is it to compare the people in your purposive sample on two measures or across different times? If so, you probably want to determine how precise you want to be able to measure this difference. If you are trying to estimate a population parameter with confidence intervals of a certain width, the sampling method may be more problematic.
Dear Roberta,
G-Power is a good tool to calculate needed sample sizes for basic statistical tests (and it is free and easy to use):
http://www.gpower.hhu.de/
The non-probabilty sample generates problems with the generalizability of your results but not with the power needed.
NON-PROBABILITY SAMPLING: This is equally acceptable method of obtaining data when the followings are observed: (i) no sampling frame; (ii) bias is negligible; (iii) sampling and sample could be monitored (chain of control and custody: who, what, when, why, and how?); (iv) probability sampling is not feasible or appropriate, i.e. if the data is time dependent and the there is a significant difference between the base period and the current period as result of time lapse (economic data); (v) data could be collected only where the collectors are available; (vi) by the circumstances and nature the sample size is small and limited, i.e. forensic evidence cases; and (vii) sampling decision is made in the field or at an operational level, i.e. QC managers select widgits from a production batch to do a "pre-test" for defects before invest resources for the "big test."
HOW TO DO NON-PROBABILITY SAMPLING?: There may be many methods used. However, these two methods may be instructive: (i) quota sampling, and (ii) representative item sampling.
(i) Quota Sampling: Given a population with non-finite size (N unknown), the researcher define all observable characteristics of the population. Determine the ration of such characteristic in the population through series of test samples. Once this proportion is known, start taking sample for the study for a "size" where the proportion of the pre-defined characteristics comes close. Here, percentage confidence becomes applicable.
(ii) Representative Item method: The researcher is looking for a particular characteristic or a set of properties (this is the purpose). The population may have characteristics: [a, b, ..., z], but the research is searching for only characters (d, e, f). First take series of sample to get an average (expected) proportion of (d,e,f) in the population, then take the "sample" needed for the study by taking the sample until the same proportion shows. This is similar to quota sampling, but in representative method, one takes look for specific characteristics: (e,d,f).
n = (Z2σ2) / SE2
... where n = minimum sample size; Z = standard score obtained by (Xi - X^) / S; σ = estimated population standard deviation; and SE = standard error where SE = σ / sqrt(n). Note that Z provides a critical value for which F(Z) = % confidence may be read from the Z-table. The fact that the sample is non-probability does not preclude us from calculation the probability among the test samples that we took. Except there is one caveat, do bootstrapping to verify if the n obtained from one trial to the next varies significantly. It is not true to say that non-probability sample precludes probability analysis.
BIAS: The concern in non-probability sampling is "bias." since the sampling is "purposive"---i.e. taking the sample for a particular purpose, we are concerned that the result may be self-fulfilling prophecy. So long as this concern may be overcome, bias minimized, then even non-probability sampling is acceptable----acceptability enough to send a person to the execution chair in case of using limited and very small genetic evidence in prosecuting a criminal in a murder case. Therefore, if non-probability sample is used, the researcher must be able to demonstrate safeguarding measures against bias. In forensic evidence, this is generally accomplished through the chain of custody.
RELIABILITY: Reliability is defined as the ability to reproduce the same result in subsequent studies following the same procedures used in the prior study. If the result could not be reproduced, the prior study fails on reliability test. Whatever sample method used, if subsequent independent study could reproduce the result it is said that the study is reliability. In genetic test, researchers are provided with limited amount of sample, generally small amount. Regardless of how the genetic material was taken, probability or non-probability sampling, if subsequent test does not reproduce the same result, the conclusion of the prior study is not reliable and, therefore, unreliable.
SAFEGUARD BY DISTRIBUTION TEST: As a precaution use distribution verification to verify if the population is normally distributed. If it is normally distributed, one may safely say that the elements in the population have equal chance of being selected. If all elements in the population exists in a state of random (equal chance of being selected), no matter if we "conveniently" or "randomly" select the elements, we should get the same result. Distribution test may be accomplished by: (i) Shapiro-Wilk Test, (ii) Cramer-von Mises Test, or (iii) Anderson-Darling Test. if the test shows that the population is normally distributed, then all elements in the set has equal chance. If the distribution test shows that the data are not normally distributed, use the chi-square test as back up for your significance test.
VALIDITY: Validity refers to the precision between the expected and the observed. in the genetic testing example, since the genetic make up of everyone is different, the result of the test----even with minute sample---must hit the mark in every test. If one test identifies the owner of the gene to be Mr. A and on subsequent test shows that the gene comes from Mr. B, the test is not valid---here, the choice of sampling method (probability or otherwise) becomes moot.
REFERENCES:
[1] Laplace, Pierre-Simon de (1774). “Mémoire sur la probabilité des causes par les événements.” Mémoires de l'Académie royale des Sciences de Paris (Savants étrangers), tome 6: 621–656.Translated by Stephen M. Stigler in Statistical Science 1 (3), 1986
[2] Lexis, Wilhelm (1878). “Sur la durée normale de la vie humaine et sur la théorie de la stabilité des rapports statistiques.” Annales de démographie internationale (Paris) II: 447–462.
[3] Lukacs, Eugene; King, Edgar P. (1954). “A Property of Normal Distribution.” The Annals of Mathematical Statistics 25 (2): 389–394.
[4] Marsaglia, George; Tsang, Wai Wan (2000). “The Ziggurat Method for Generating Random Variables.” Journal of Statistical Software 5 (8).
[5] Marsaglia, George (2004). “Evaluating the Normal Distribution.” Journal of Statistical Software 11 (4).
[6] Papoulis, A. Probability, Random Variables, and Stochastic Processes, 2nd ed. New York: McGraw-Hill, pp. 100-101, 1984.
[7] Rohrbasser, Jean-Marc; Véron, Jacques (2003). “Wilhelm Lexis: The Normal Length of Life as an Expression of the “Nature of Things.”” Population 58 (3): 303–322.
[8] Stigler, Stephen M. (1982). “A Modest Proposal: A New Standard for the Normal.” The American Statistician 36 (2): 137–138.
[9] Wallace, C. S. (1996). “Fast pseudo-random generators for normal and exponential variates.” ACM Transactions on Mathematical Software 22 (1): 119–127.
[10] Whittaker, E. T. and Robinson, G. “Normal Frequency Distribution.” Ch. 8 in The Calculus of Observations: A Treatise on Numerical Mathematics, 4th ed. New York: Dover, pp. 164-208, 1967.
Your sample size will be based on the statistical technique you plan to perform, and the statistical precision, or power you want to have. There's a little more to it than that, but once you decide the test or confidence interval you're going to calculate you can solve for n to see what your recommended sample size is. You'll likely want to do this across a range of assumptions.
Roberta -
Are you using continuous data? Do you have any related census that could be used as regressor data? If you do, you could try the following:
https://www.researchgate.net/publication/261947825_Projected_Variance_for_the_Model-based_Classical_Ratio_Estimator_Estimating_Sample_Size_Requirements
Cheers - Jim
PS - Note that whatever you do, it is always good to group data as homogeneously as possible, treating the groups separately, whether we mean strata, or applications for models, or response propensity groups, whatever, even if you have to post-stratify.
PSS - The "range of assumptions" Matt noted may be key to whatever you do. Careful collection of metadata and documentation of your methodology, including all metadata, may help make your study more useful in concert with other studies.
Conference Paper Projected Variance for the Model-based Classical Ratio Estim...
the calculation of sample size depends on the hypothesis or research question, and not on the probability or non probability.
To do power analysis to estimate your sample size, you have to write your hypothesis, and based on that you decide what statistical test you will use. It should be one of the inferential statistics. so you need to determine the following: alpha {standard to be .05}, power [standard to be .80], effect size {small, moderate, or large, each test has its own value, you can find these values in the net}. Then download free programs to calculate the sample size such as G. power.
Muayyad
That is very insightful. I am also having difficulty with determining how I should approach a power analysis for a convenience sample.
Dear Alisha,
if you send me or write here your research hypothesis, this will help in figuring out which test to use and to determine the sample size as well.
my email: [email protected]
The level of satisfaction and the rate of clinical documentation compliance in perioperative nurses will increase with a clinical quality metrics dashboard intervention.
H0: the level of satisfaction in perioperative nurses using an EHR for clinical documentation is related to an increase in the rate of documentation compliance
H1: the level of satisfaction in perioperative nurses using an EHR for clinical documentation is not related to an increase in the rate of documentation compliance
Hi Robert
Did you get a proper answer for this question? How did you calculate the sample size at the end? I have the same problem. I am going to use voluntary and snowball sampling. Would you please help me in that?
Best
There is a great deal of good work on nonprobability sampling to be found on Mike Brick's ResearchGate pages. See https://www.researchgate.net/profile/J_Brick
But you cannot just pretend it is random sampling and apply some hypothesis test (even if you did the 'test' properly with a type II error probability analysis). I saw one paper somewhere which - through some logic I don't recall - said a snowball sample had to be about twice as large as a random sample, as I recall ... I think, and I found some other paper on snowball sample sizes very quickly online which may be very good, but I dont know enough about it to recommend it. But just search, and pay attention to which sources seem reliable. -
I think that bias is a problem that could be your biggest problem. You could take a relatively huge sample size, but if there is selection bias, your results could be very misleading.
Have you researched "response propensity" groups? If not, you might want to do that.
The Pennsylvania State University has a good deal of good information online, available to all, and they use a textbook by Thompson which I THINK, might have something on such sampling, but I don't know for sure.
Didn't network sampling start with Monroe Sirken?
Best wishes in your research.
How do I calculate the sample size of the study of snowball? The study is quantitative and the sample strategy is non-probability.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Statistical Techniques