In finite population survey statistics, for survey methodology and data analysis, data are stratified to reduce overall variance. But sometimes publishing the individual categories becomes important, and this may negate the role of reducing overall variance for a given overall sample size. Further, the categories chosen may not be best for stratification purposes. In the case of regression model-based methods, the goal is the same. In that case, scatterplots and estimated regression coefficients with their standard errors can be used to sort out which data should go into which strata. Thus, regression analysis is important for model-based sampling and/or estimation by prediction.
For design-based sampling and estimation, there is Neymann allocation, but my question is not so much about allocating to strata already defined, but more how to define the strata in the first place. There must be some categorical-type heading, but one might do better by being imaginative as to what such data groupings could be tried.
Sometimes better ways to stratify become apparent after-the-fact, and poststratification is used.
What tips/methods do you propose, and/or examples do you have for stratifying either for design-based sampling and estimation methods, or model-based methodology, or model-assisted design-based methods?
Guillermo -
You said that "Sampling is a Science and an Art...." Nicely said. I very much agree.
At the end of your response, with regard to allocation, you have "...but it may be different with each of the variables, then you have to order the variables by their importance and select the allocation generally more suitable." Yes, that would be unavoidable with regard to stratum selection. Good point, appropriate to this question. It also reminds me that at the estimation stage, if a PPS sample had been drawn, then the same kind of compromise or attention to the most important items/target variables would need to have been done at the data selection stage, that would impact the estimation. Here I refer to the measure of 'size' used, which would be attached to each respondent for every response item. However, if model-based estimation ('prediction' from regression) were used, then each item could have its own size measure in the regression weights. But it does get more complex when design weights and regression weights are combined in 'calibration' weights.
So, I had thought of prediction/model-based estimation as avoiding having to compromise or prioritize your survey questions/items, but you have a good point that for stratification, the very question at hand, this kind of compromise or prioritization still occurs.
It is interesting to consider how your sampling method and your estimation interrelate.
Thank you - Jim
Dear James
When stratifying you have to take care at once of all the aspects that you mentioned. It is better that the categorical variables for stratification, were the same that those of your domain of study, then you are in better condition to produce small variance estimators, but in many studies the categorical variables that define the domain of study, are not known on the sample frame, and they only may be ascertained after the sample is done, then you may use poststratification. Sampling is a Science and a Art, then the particular preference of the designer have a place in the selection of the strata too. For the same problem you may get two or more very good sampling designs with relative adventajes between them, but without any of them were clearly the best. If you have a multiple stage design with stratification, the strata selection is more critical for the sampling of the primary units, but you may have stratification in each of the stages. Finally the allocation of the sample to the strata may be done with Neymann method but it may be different with each of the variables, then you have to order the variables by their importance and select the allocation generally more suitable.
Guillermo -
You said that "Sampling is a Science and an Art...." Nicely said. I very much agree.
At the end of your response, with regard to allocation, you have "...but it may be different with each of the variables, then you have to order the variables by their importance and select the allocation generally more suitable." Yes, that would be unavoidable with regard to stratum selection. Good point, appropriate to this question. It also reminds me that at the estimation stage, if a PPS sample had been drawn, then the same kind of compromise or attention to the most important items/target variables would need to have been done at the data selection stage, that would impact the estimation. Here I refer to the measure of 'size' used, which would be attached to each respondent for every response item. However, if model-based estimation ('prediction' from regression) were used, then each item could have its own size measure in the regression weights. But it does get more complex when design weights and regression weights are combined in 'calibration' weights.
So, I had thought of prediction/model-based estimation as avoiding having to compromise or prioritize your survey questions/items, but you have a good point that for stratification, the very question at hand, this kind of compromise or prioritization still occurs.
It is interesting to consider how your sampling method and your estimation interrelate.
Thank you - Jim
Regarding poststratification:
Whether design-based, model-based, or model-assisted design-based sampling, poststratification can be helpful. Survey statisticians often use it for design-based sampling, but also consider the following application for strictly model-based sampling and estimation: In the paper found through the link below, in section 4, " What if the smallest members of the population may behave very differently?" there is a note regarding stratification. An example was a monthly sample survey of production for natural gas wells, with regressor data from an annual census survey. It appeared that some smaller wells were behaving differently from the others, with regard to the ratio regression coefficient, and that size stratification might help. However, further information revealed that there were two kinds of wells involved: traditional and shale oil. Stratification by well type would likely be best. Depth of the well is a good proxy to determine well-type. Even if the delineation is not perfect, the idea is stratification to improve accuracy of estimated totals for overall totals by geographic area, not to publish by well-type. This should reduce variance overall, as is the goal with stratification in general, but also, here it would reduce bias due to 'model-failure' (a term to be evaluated in the context of overall/total survey error (TSE)).
Article Efficacy of Quasi-Cutoff Sampling and Model-Based Estimation...
See the following two papers if you can understand:
Singh, R. (1971) Approximately optimum stratification on the auxiliary variable, J. Amer. Statist.Assoc., 68, 829-833.
Singh, Ravindra and Sukhatme, B. V. (1969) Optimum stratification, Ann. Inst. Statist. Math.,21, 515-528.
For studying school achievement,
http://betterevaluation.org/evaluation-options/stratifiedrandom
A possibility for building strata based on an auxiliary variable is to divide the total range of the auxiliary variable in a number L of different consecutive intervals. Each interval contains the units of a stratum with auxiliary variable belonging to the interval. This method is very good when there is high or low correlation between the interest and the auxiliary variables. The theory was studied by Tore Dalenius (1957). Sampling in Sweden. Stockholm.
Other reference is: G. J. Glasser (1962). Rev. Internat. Statist. Inst.
Nous utilisons souvent des méthodes de classification automatique qui nous permettent de prendre en charge plusieurs variables simultanément. Il est évident que les variables doivent être de même nature. la question qui se pose alors c'est celui du choix de la méthode d'agrégation et la distance à utiliser. les logiciels proposent plusieurs méthode. Dans notre cas nous avons souvent utilisé la méthode de WARD.
Using "Google translate" for the above for 'French to English,' we see this from Nacer-Eddine:
"We often use automatic classification methods that allow us to support multiple variables simultaneously. It is obvious that the variables must be of the same nature . the question that arises is that the choice of the method of aggregation and remote use . software provide several methods. In our case we have often used the method of WARD ."
I also Googled WARD, and it seems to be for cluster analysis. This sounds like a good approach, the idea being to reduce variance within clusters/strata while making the difference between strata larger.
Effectivement, il s'agit bien de cluster analysis. Je profite pour remercier James pour sa traduction. En réalité parmi les méthodes de classification nous distinguons celles descendantes (comme la segmentation) et celles ascendantes. Dans notre cas nous avons utilisé la classification ascendante hiérarchique (CAH). Nous l'avons beaucoup plus utilisé dans les enquêtes auprès des ménages (household survey) où nous disposions de taux (équipement, chômage, scolarisation, activité, etc.) mais aussi d'autres variables telles que la taille moyenne des ménages ou l'indice synthétique de fécondité (fertility rate). dans ces cas les logiciels permettent d'utiliser les variables brutes ou les transformer (par exemple les Zscores). la question dès lors est de savoir le type de transformée à utiliser.
Randomization, as Ken Brewer has written, may be conceived by the public as "fair." There is no doubt that it can reduce bias that could occur with "purposive" sampling in some applications. However, as Abhijit noted, we pick strata (and I'll add we also pick data grouping categories for small area estimation) purposefully. This is based on our knowledge or assumption that some categories of data will have a smaller within stratum variance, and the differences between strata will be increased, if we choose the categories that we do purposefully choose. Often these categories are basically picked for us, such as lower level geographic data within a larger geographic area. (These natural categories may also need to be published separately, which generally requires a larger sample than if this is not the case.) We may, however, find that better categories, for estimation purposes, can be purposefully chosen, based on study of these data, and/or subject matter theory.
"We may, however, find that better categories, for estimation purposes, can be purposefully chosen, based on study of these data, and/or subject matter theory."
For purposes of applying models, availability of good regressor data is important. As with strictly random sampling and estimation, categories for sampling and estimation using models can also be purposefully chosen to reduce variance, just as in stratified random sampling. But with model-based estimation and sampling, this also will often substantially reduce the bias. This is also true for small area estimation, as illustrated in graphics in the paper found through the first attached link. There, instead of observing what data may be split for application of different models, we are graphically observing what categories may be combined under a model. (A graph from that first paper is attached.) The purpose is still, as discussed in the RG abstract for the paper found through the next link, and the paper found through the link after that, to determine what data should be modeled together, and what should not be modeled together, but in separate groups or strata. (Modeling can be used to assist design-based methods, but strictly model-based methods for highly skewed populations can be used to avoid the pitfalls of PPS sampling - see Basu's Elephant Fable, for example - and avoid extremely inefficient use of resources - see examples in next link. Note also that the size measure is by attribute, not just one-size-fits-all, as in PPS sampling and estimation.)
Article Quasi-Cutoff Sampling and Simple Small Area Estimation with ...
Article Efficacy of Quasi-Cutoff Sampling and Model-Based Estimation...
Article On Model-Failure When Estimating from Cutoff Samples
Conference Paper Projected Variance for the Model-based Classical Ratio Estim...
Supply Chain Materiel Management Procedures - Metrics and Inventory Stratification Reporting
A uniform portrayal of requirements and assets application that is a computer-generated, time-phased simulation of actions causing changes in the supply position; e.g., procurement, repair, receipt, issue, termination, and disposal of materiel.
http://www.dtic.mil/whs/directives/corres/pdf/414001m/414001m_vol10.pdf
Please. There is no need to downvote because a valued RG contributor may have misunderstood a question. (I was politely informed of such a mistake that I made previously, and I appreciated that.)
The downvoting option on RG should be eliminated.
Thank you.
The two major types of stratification, it seems, are (1) subject matter category, and (2) size.
I suppose one might do some kind of cluster analysis as a third 'type,' but that sounds too arbitrary to me. However, perhaps someone else may have found that useful. Does anyone have such an application to share? Such a third type, as well as stratification by a size measure, sound like stratification which could improve overall accuracy, and, for stratified random sampling, warrant the use of Neyman allocation, whereas the first one may call for subtotal estimates, making it a different problem. Or not.
I think Ken Brewer (Australia) once said that his mentor, Ken Foreman, said that there was 'no substitute for knowing your data.'
Has anyone used cluster analysis to define strata? Did it turn out that the 'clusters' were actually categories re the subject matter?
- Badges
- Science topic