A number of people have asked on ResearchGate about acceptable response rates and others have asked about using nonprobability sampling, perhaps without knowing that these issues are highly related. Some ask how many more observations should be requested over the sample size they think they need, implicitly assuming that every observation is at random, with no selection bias, one case easily substituting for another.
This is also related to two different ways of 'approaching' inference: (1) the probability-of-selection-based/design-based approach, and (2) the model-based/prediction-based approach, where "prediction" means estimation for a random variable, not forecasting.
Many may not have heard much about the model-based approach. For that, I suggest the following reference:
Royall(1992), "The model based (prediction) approach to finite population sampling theory." (A reference list is found below, at the end.)
Most people may have heard of random sampling, and especially simple random sampling where selection probabilities are all the same, but many may not be familiar with the fact that all estimation and accuracy assessments would then be based on the probabilities of selection being known and consistently applied. You can't take just any sample and treat it as if it were a probability sample. Nonresponse is therefore more than a problem of replacing missing data with some other data without attention to "representativeness." Missing data may be replaced by imputation, or by weighting or reweighting the sample data to completely account for the population, but results may be degraded too much if this is not applied with caution. Imputation may be accomplished various ways, such as trying to match characteristics of importance between the nonrespondent and a new respondent (a method which I believe has been used by the US Bureau of the Census), or, my favorite, by regression, a method that easily lends itself to variance estimation, though variance in probability sampling is technically different. Weighting can be adjusted by grouping or regrouping members of the population, or just recalculation with a changed number, but grouping needs to be done carefully.
Recently work has been done which uses covariates for either modeling or for forming pseudo-weights for quasi-random sampling, to deal with nonprobability sampling. For reference, see Elliott and Valliant(2017), "Inference for Nonprobability Samples," and Valliant(2019), "Comparing Alternatives for Estimation from Nonprobability Samples."
Thus, methods used for handling nonresponse, and methods used to deal with nonprobability samples are basically the same. Missing data are either imputed, possibly using regression, which is basically also the model-based approach to sampling, working to use an appropriate model for each situation, with TSE (total survey error) in mind, or weighting is done, which attempts to cover the population with appropriate representation, which is mostly a design-based approach.
If I am using it properly, the proverb "Everything old is new again," seems to fit here if you note that in Brewer(2014), "Three controversies in the history of survey sampling," Ken Brewer showed that we have been all these routes before, leading him to have believed in a combined approach. If Ken were alive and active today, I suspect that he might see things going a little differently than he may have hoped in that the probability-of-selection-based aspect is not maintaining as much traction as I think he would have liked. This, even though he first introduced 'modern' survey statistics to the model-based approach in a paper in 1963. Today it appears that there are many cases where probability sampling may not be practical/feasible. On the bright side, I have to say that I do not find it a particularly strong argument that your sample would give you the 'right' answer if you did it infinitely many times when you are doing it once, assuming no measurement error of any kind, and no bias of any kind, so relative standard error estimates there are of great interest, just as relative standard error estimates are important when using a prediction-based approach, and the estimated variance is the estimated variance of the prediction error associated with a predicted total, with model misspecification as a concern. In a probability sample, if you miss an important stratum of the population when doing say a simple random sample because you don't know the population well, you could greatly over- or underestimate a mean or total. If you have predictor data on the population, you will know the population better. (Thus, some combine the two approaches: see Brewer(2002) and Särndal, Swensson, and Wretman(1992).)
..........
So, does anyone have other thoughts on this and/or examples to share for this discussion: Comparison of Nonresponse in Probability Sampling with Nonprobability Sampling?
..........
Thank you.
References:
Brewer, K.R.W.(2002), Combined Survey Sampling Inference: Weighing Basu's Elephants, Arnold: London and Oxford University Press
Brewer, K.R.W.(2014), "Three controversies in the history of survey sampling," Survey Methodology, Dec 2013 - Ken Brewer - Waksberg Award:
https://www150.statcan.gc.ca/n1/pub/12-001-x/2013002/article/11883-eng.htm
Elliott, M.R., and Valliant, R.(2017), "Inference for Nonprobability Samples," Statistical Science, 32(2):249-264,
https://www.researchgate.net/publication/316867475_Inference_for_Nonprobability_Samples, where the paper is found at
https://projecteuclid.org/journals/statistical-science/volume-32/issue-2/Inference-for-Nonprobability-Samples/10.1214/16-STS598.full (Project Euclis, Open Access).
Royall, R.M.(1992), "The model based (prediction) approach to finite population sampling theory," Institute of Mathematical Statistics Lecture Notes - Monograph Series, Volume 17, pp. 225-240. Information is found at
https://www.researchgate.net/publication/254206607_The_model_based_prediction_approach_to_finite_population_sampling_theory, but not the paper.
The paper is available under Project Euclid, open access:
https://www.doi.org/10.1214/lnms/1215458849
Särndal, C.-E., Swensson, B., and Wretman, J.(1992), Model Assisted Survey Sampling, Springer-Verlang
Valliant, R.(2019), "Comparing Alternatives for Estimation from Nonprobability Samples," Journal of Survey Statistics and Methodology, Volume 8, Issue 2, April 2020, Pages 231–263, preprint at
https://www.researchgate.net/publication/337950671_Comparing_Alternatives_for_Estimation_from_Nonprobability_Samples
This is a very interesting perspective, James R Knaub , and one that you could well share on Frank Harrell's Datamethods discussion forum : https://discourse.datamethods.org
Other than that, I'm going to have a look at those references over a largeish pot of coffee before I say anything stupid (stupid plus references allows you to cover your retreat better!)
r
The topic may have been more clearly stated as follows:
"Comparison of Nonprobability Sampling with Nonresponse in Probability Sampling"
Or
"Comparison of Nonprobability Sampling with Nonresponse in Probability Sampling, Regarding Inference"
Sometimes random sampling is not practical for various reasons. Work such as Valliant, R.(2019), "Comparing Alternatives for Estimation from Nonprobability Samples," Journal of Survey Statistics and Methodology, Volume 8, Issue 2, April 2020, Pages 231–263, preprint at
https://www.researchgate.net/publication/337950671_Comparing_Alternatives_for_Estimation_from_Nonprobability_Samples,
found in the discussion question above, may help address that.
In the case of imputation, one can think of prediction (regression) as a way to 'fill in' for any data not collected. That would then be a model-based approach. At the US Energy Information Administration, very simple prediction, ratio models, are often used to 'fill in' for data not collected, often or even usually in the case of very small establishments which can not easily supply high quality data frequently. See the following comment I wrote in this regard:
https://www.researchgate.net/publication/340608318_Sampling_and_Estimation_from_Finite_Populations/comments
Thank you.
- Badges
- Science topic
- Similar topics
- Chemistry
- Biochemistry
- Biochemical Techniques
- Blotting Techniques
- Immunoblotting
- Dyes