Thank you Ali. Your sites express that probability sampling is a requirement for inferential analysis.

Thank you Ali. Your sites express that probability sampling is a requirement for inferential analysis.

Inferential statistics are used to make generalizations from a sample to a population. Thus, dear Eddie, inferential statistics, samples are required.

probability sampling is essential for inferential analysis. In order to minimize the cost the Inferential statistics done first and there after they are used to make generalizations from a sample to a population.

Inferential statistics - as its name implies, is to conclude / infer some conclusions based on the empirical findings generated. 

To embark on an inferential research for the purpose of generalization, a study is conducted on random sample (or sample with normal distribution) extracted from the entire population.  Significant results from the random sample (after running some parametric statistical tests / analyses) can then generalize to the population.  Hence, inferential / parametric statistical analysis required probability sample.

If you don't have probability / random sample in which you only have some convenient / snow-ball / purposive sample or the distribution of the sample is not-normal, you still can use inferential statistics but those inferential statistics is called non-parametric statistics (sometimes also called non-parametric inferential statistics).  If non-parametric statistics is used, you can only conclude the findings based on the sample collected and can't generalize to the entire population.

Yes my Dear Eddie Seva See;  probability sample required in inferential statistical analysis. Please go through the links attached.

https://extension.usu.edu/evaluation/files/uploads/Start%20Your%20Engine/Study%20the%20Route/Analyze%20the%20Data/UsingInferentialStatistics.pdf

http://www2.fiu.edu/~ganapati/4153/regression.html

Thank you Sarwan. Your slides and other documents indeed show that probability sampling is required in inferential analysis. Ed

Eddie -

There are three basic methodologies for inferring to a finite population from a sample. Two use randomization in some way, and one does not need it, as long as any necessary stratification is included, and there is good regressor data, often from a related census survey already available.

Design-based (randomized/probability) sampling and estimation has been used a great deal since the 1940s. These are familiar techniques such as simple random sampling, stratified random sampling, and cluster sampling, found in a number of texts, including Cochran, W.G.(1977), Sampling Techniques, 3rd ed., John Wiley & Sons, and Lohr, S.L.(2010), Sampling: Design and Analysis, 2nd ed., Brooks/Cole. Others include one by Hansen, Horwitz, and Madow, and by Leslie Kish.

Model-assisted design-based sampling uses auxiliary data (actually regressor data) to help compensate for random draws that did not turn out to be very 'representative,' and can greatly improve results. Cochran includes regression and ratio estimation of this sort. Lohr and others also do this. Here is a particularly good reference for this second general methodology:

Särndal, C.-E., Swensson, B. and Wretman, J. (1992), Model Assisted Survey Sampling, Springer-Verlag.

Regression alone, however, with proper stratification or categorization to ensure data 'covered' by a given model all 'belong' there, has been used with a great deal of success. I have used it for highly skewed establishment surveys with results that tested out very well and held up well for decades. I was a little more surprised not long ago to converse with a researcher in England who had had good use of it with household surveys.

In Cochran, section 6.7, "Conditions Under Which the Ratio Estimator is a Best Linear Unbiased Estimator," pages 158-160, he describes work done independently by Ken Brewer, and Richard Royall, that was basically for cutoff sampling. In follow-up years, Richard Royall advocated "balanced" sampling, where the mean of the regressor data for the sample should be the same as the mean of the regressor data for the population. A number of survey statisticians have written on such balanced sampling. Ken Brewer went on to combine randomization and modeling in articles and in his book: Brewer, KRW (2002), Combined survey sampling inference: Weighing Basu's elephants, Arnold: London and Oxford University Press.

Around 1990, I heard a seminar where Nancy Kirkendall discussed the use of model-based (regression/prediction) methods for inference to finite populations, based on what she and Yahia Ahmed had written on official statistics, based on what they had learned from Royall's work (apparently not knowing of Ken Brewer's earlier work). I developed quasi-cutoff sampling with regression for inference for about a quarter of a century since then, with the help and collaboration of other statisticians, and results have been very successful. There are various ways of proving this to varying degrees. See the paper at the first attached link. There are some useful definitions given there also. I also have a number of other papers in this area under my profile.

So, though I would say most survey statisticians rely on randomization, it does not always give you the best results. Statisticians like Ken Brewer and others have relied on randomization, but also include an element of modeling (regression). Others may be happy enough with balanced sampling, as being superior to random sampling, and Ken has written on balanced sampling as well. However, a very simple example at the second link below makes a comparison between quasi-cutoff sampling and balanced sampling that argues well for quasi-cutoff sampling in many cases for establishment surveys. The final link below is for a presentation made by Joel Douglas and me in 2010. This methodology has been applied to a number of surveys at the US Energy Information Administration. I am now retired, but work continues there to expand this methodology for efficient use of resources, and I have been told that it is planned that work I have uploaded here on RG will continue to be heavily referenced. Some of it has also been referenced around the world in several languages.

Please note that nonsampling error, such as measurement error, is a very important consideration. Data quality considerations will impact your ability to succeed with any survey. Thus you should consider "total survey error" when deciding on sampling methodology, and all of the kinds of error that may occur. This should influence your decision as to what methodologies may provide the more accurate inferences.

Cheers - Jim  

Article Efficacy of Quasi-Cutoff Sampling and Model-Based Estimation...

Conference Paper Projected Variance for the Model-based Classical Ratio Estim...

Presentation Cutoff Sampling and Estimation for Establishment Surveys

I agree with all them that  to avoid biased and to be sure that the individuals

included in a sample represent a cross section of individuals in

the population  the probability sample is  required in inferential statistical analysis and thank for all of you for the nice references.  

If you have auxiliary/regressor data on the finite population, then even limiting yourself to the arguments of randomization, balanced sampling would have to be better instead of completely random sampling (for either simple or stratified cases). Please do an Internet search on "balanced sampling ."

Also, you might find this ISR article by Ken Brewer to be of interest :

http://onlinelibrary.wiley.com/doi/10.1111/j.1751-5823.1999.tb00379.x/abstract

See chapter four in

Lohr, S.L.(2010), Sampling: Design and Analysis, 2nd ed., Brooks/Cole, as well as 6.7 in Cochran (above ).

Also see these

https://www.researchgate.net/publication/261474011_The_Classical_Ratio_Estimator_%28Model-Based%29

https://www.researchgate.net/publication/242677296_Quasi-Cutoff_Sampling_and_Inference

And

Supporting Statements, to Office of Management and Budget, for Survey Clearance

Some information provided to the Office of Management and Budget by various US Energy Information Administration personnel, with regard to use of quasi-cutoff sampling and model-based estimation, is found at the URLs below. More applications should follow in subsequent years.

http://www.reginfo.gov/public/do/DownloadDocument?documentID=186666&version=2, electric power data, re “EIA-826, EIA-923,” 2011

http://www.reginfo.gov/public/do/DownloadDocument?documentID=277030&version=2, natural gas receipts, re “EIA-857,” 2011 (revisions needed)

http://www.reginfo.gov/public/do/DownloadDocument?documentID=355180&version=0, electric power data, re “EIA-923,” 2012

http://www.reginfo.gov/public/do/DownloadDocument?documentID=381390&version=0, natural gas and oil reserves, re “EIA-23,” 2013

For further information on US Energy Information Administration (EIA) surveys, see these web pages: http://www.eia.gov/survey/.

See also

https://www.researchgate.net/publication/261474159_Some_Proposed_Optional_Estimators_for_Totals_and_their_Relative_Standard_Errors_for_a_set_of_Weekly_Quasi-Cutoff_Sample_Establishment_Surveys

https://www.researchgate.net/publication/261586154_Using_Prediction-Oriented_Software_for_Survey_Estimation

https://www.researchgate.net/publication/261588075_Using_Prediction-Oriented_Software_for_Survey_Estimation_-_Part_III_Full-Scale_Study_of_Variance_and_Bias

... and so on.  

Article The Classical Ratio Estimator (Model-Based)

Article Quasi-Cutoff Sampling and Inference

Article Some Proposed Optional Estimators for Totals and their Relat...

Article Using Prediction-Oriented Software for Survey Estimation

Article Using Prediction-Oriented Software for Survey Estimation - P...

In short, the answer to "Is a probability sample required in inferential statistical analysis?" is "No, not always." This is explained in my previous posts above and backed by information regarding articles, books, and researchers, and by many years of application, producing thousands of published numbers (estimated totals, estimated standard errors, ...) for Official Statistics. Model-based estimation is used rather than design-based estimation. When this can be done, I highly recommend one examine scatterplots for data editing purposes, and to gain insight into your data, its uses and its limitations. Thank you. 

Thanks Dr. Kaub for the share

Thank you James for your comprehensive view and documents. The following document might make you entertain second thoughts about the matter. Ed

http://link.springer.com/article/10.1007%2Fs10739-006-9119-z

James, these are some of the textbooks that concur with or forward Fisher's opinion on randomization. Ed

Burns, Nancy & S.K. Grove. 1997. The Practice of Nursing Research Conduct, Critique and Utilization. 3rd ed. W.B. Sande4rs Co. The Curtis Center Independence Square West Philadelphia, Pennsylvania 19106.

Davis, Duane, J. Utts & M. Simon, 2002. Statistics and Research Methods for Managerial Decision. Second Edition. Brrok-Cole. 511 Forest Lodge Road. Pacific Grove, CA 93950, USA.

Mc Daniel, Carl & R. Gates. 2002. Marketing Research: The Impact of the Internet. Southwestern Thomson Learning 5101 Madison Road, Cincinnati, Ohio 45227. http://www.swcollege.wm

Pelosi, Marilyn, T.M. Sandifer, & U. Sekaram. 2001. Research and Evaluation for Business. John Wiley and Sons, Inc. 605 3rd Vaenue, NY, NY.

James,this information might also help persuade you

Appropriate statistical applications are important to research. This matter has been classically illustrated in the case of the presidential elections in the United States of America in 1936. The Literary Digest, with non-random selected respondents of 2,376,523 wrongly predicted that Landon would win over Roosevelt, while George Gallup, with a low but randomly selected voters of 3,000, correctly foretold that Rossevelt would win (Sprinthall, 1997).

Sprinthall, Richard. 1997. Basic Statistical Analysis. Fifth Edition. Allyn and Bacon. Boston.\

Subrata, I am sure our friend is Dr. Knaub, not Dr. Kaub. Peace. Ed

Hi Eddie -

Actually you cannot really argue with success, and i have had plenty for a long time, with establishment surveys. In actual practice, results have been outstanding - for many, many results. It depends upon the circumstances.

I am very, very well aware of all of the points-of-view and do have them in my repertoire. But my experience is that those who have not used model-based estimation, generally are not familiar with it, and those adamantly opposed to using it with quasi-cutoff sampling in an establishment survey environment, have generally not actually tried it. If you take a "total survey error" approach, you will see that it avoids data quality issues that can very much invalidate a random sampling approach, in some circumstances. This I know for certain.

The biggest issue is one Ken Brewer noted once: People think of random sampling as 'fair.' That is a big reason for its popularity and why statisticians drift to it - besides often being bullied if they don't. But I have found that too much is assumed when people do not check into the actual applications and what is practical, and compare how well different approaches actually perform for different circumstances.

As an aside: Note that in general, model-assisted design-based estimation is used to help rescue random sampling.

Whenever you have regressor data on a finite population, you might want to examine the following and decide for yourself, based on your own particular circumstances, what is best:

Chambers and Clark (2012),

http://ukcatalogue.oup.com/product/9780198566625.do

Valliant, Dorfman, and Royall (2000),

http://www.wiley.com/WileyCDA/WileyTitle/productCd-0471293415.html

Even Chambers, I found when talking to him at a seminar he did, refers to cutoff sampling as "extreme" sampling, and did not think much of it, though another well-known statistician came to my rescue, to a degree. :-) But my impression is that a great deal of what you read is purely theoretical, and does not account for the actual situations you will encounter in practice. For example, a 'formula' for sample size determination relies on the variance to be expected, but in an establishment survey, if you try to obtain data from a great many tiny entities, you just get awful data which inflates those variances and invalidates your sample size estimate.

Thanks for 'listening.'

Jim

James, I still am listening. Model-based design estimation, as I understand it still implies randomization. I admire the experience-based information you are providing us and it is a great thing that while you are providing the "other" side, you actually know the other "other side". Your situation is absolutely different from most researchers   "who leaped into non-randomization without knowing or even bothering to know what randomization is.: It is quite pathetic because they make your knowledge-and-experience-based-stand as the "justification", hook-line-and-sinker for their "dangerous things".

I wish and hope that your position will find its way into textbooks because we all know that textbooks are what students-future researchers- make as their "bible".

As of this date, no textbook yet will "teach" this revolutionary other side. And it will further confuse our already estranged students from the basics.

Ed

Basics?

Perhaps the best known textbook for survey statistics is Cochran, and he had all three areas in there - the three that Ray Chambers and I have independently listed as

1) design-based

2) model-assisted design-based

and

3) model-based

I can't really see Cochran as radical. Very traditional, in fact.

Your survey sampling students would do very well to start with Cochran:

Cochran, W.G.(1977), Sampling Techniques, 3rd ed., John Wiley & Sons.

Lohr and others are good too, of course.  

Yes, sir James, it's the "basics" of the many simplified versions of sampling formulae, requiring inputs of variance and other "parameters" many textbooks use. The only difference is that newer versions of text books present simpler and student-friendly approaches. I think "design-based, model-based, and model-assisted design-based are all in order" But I also believe they do not displace "ramdomization". Ed

I wanted to avoid going on here, and I guess I'll let anything else go after this, but I don't think i have been communicating very well here, so two more things up front:

1) randomization means design -based

2) there is a big difference between nonprobability sampling with or without regressor data, properly stratified, which is where the inference is derived (see "variance of the prediction error")

- Thus the Roosevelt election example is completely off topic here. It is something we have used as a very good example of the problem with such "surveys," but it is certainly not what I was talking about at all. There is no regression or ratio model used there. It is not part of anything I was saying. It is a classic example of not using randomization, and it did not use modeling of any useful kind either, so it was of course flawed. That kind of survey needs a randomized design. (Of course you also have to deal with nonresponse bias, perhaps through response propensity groups.)

Yes, I dealt with various surveys for a long time. Randomization is certainly an important topic to begin with, and very useful, but models are part of all statistics, and regression/ratio models are certainly important in survey statistics. - One of the definitions Sage contracted me to write for one of their encyclopedias was for "survey weights," a completely randomized technique, which is on my RG page. Then you add in the auxiliary/regressor data and you get to calibration weights, which are a combination of survey weights and regression weights, well accepted in survey statistics, noted elsewhere on my page where I wrote about the various types of ratio estimates in use for official statistics at a statistical (survey) agency.

So my point is not to take away from random sampling and related estimation (which is design-based), but to point out the various uses for models (regression), which can, in certain cases, become more important.

But yes, by all means, teach your students about randomization. That is fine, and necessary, but it is just the beginning.

So I am absolutely not saying randomization isn't an important concept. Perhaps now you might re-read my first answer/post here.    

Got it James. Thanks for the knowledge shared with all of us here in this discussion. Ed

Oops. Sorry. Another use of various types of modeling for inference for finite population surveys that I should have mentioned is "small area estimation" (SAE),  There you "borrow strength" (cautiously!) from other related data. There is much literature on this. Notably, consider the work of J.N.K. Rao.  -  It is more likely to be problematic than the other things we discussed, but it is in widespread and successful use. 

The first link below is to information on a popular book by JNK Rao (Small Area Estimation, Wiley 2003) which includes Bayesian models.  The second is for more information on SAE modeling, and the last two are papers I wrote (2014 and 1999, the 1999 one having been used extensively and still in use).  My two papers here are rather straightforward in using "borrowed" data and do not include Bayesian methods.  

This is why I have to say that randomization, though important, is not always essential to inference, including estimating standard errors. But if you do not randomize, you must use regression modeling of some sort. Combining the two (randomization and modeling) is often best.

SAE is a big part of survey statistics, and taught in university and other courses. - Thanks. 

http://www.wiley.com/WileyCDA/WileyTitle/productCd-0471413747.html

http://projecteuclid.org/euclid.ss/1177010647

http://www.amstat.org/sections/SRMS/Proceedings/papers/1999_115.pdf

Article Quasi-Cutoff Sampling and Simple Small Area Estimation with ...

Another nonprobability SAE example on page 168, example 5.2 in 

http://js.iasri.res.in/jisas/jsp/volume/vol57/jnkrao.pdf

and as a different kind of example:

Karmel, T.S., and Jain, M. (1987), "Comparison of Purposive and Random Sampling Schemes for Estimating Capital Expenditure," Journal of the American Statistical Association, Vol.82, pages 52-57.

Note Karmel and Jain's conclusion that they were expecting the purposive sampling estimates to be worse, but instead they were very much better  than the random sampling based estimates.  The reason is that they used regression model-based estimation and stratification. 

FYI: New book for "students and researchers," by Edward and Johnny Blair:

Applied Survey Sampling.

Attached is a part of the table of contents I found online. 

Consider this paper:

Kirkendall, N.J. (1992), “When Is Model-Based Sampling Appropriate for EIA Surveys?” from the 1992 American Statistical Association (ASA) Proceedings for the Survey Research Methods Section, found at

https://www.amstat.org/sections/SRMS/Proceedings/papers/1992_107.pdf

pp. 637-642.

This quote from the beginning of that paper is relevant:

"There are survey situations in which a model-based sample

(perhaps a multiattribute cut-off sample) can result in greater accuracy for estimating totals than can be derived from a traditional probability based sample of the same size."

However, I think the point of the question here was that a nonprobability sample cannot be used to make inferences as if it were a probability sample - that is, without judicious use of regression modeling and data grouping. Further, though model-assisted methods are fairly common, strictly model-based sampling is not. Thus, probability sampling is far more commonly used to obtain reliable inference. When there is nonresponse, the imputation or survey reweighting that occurs causes one not to strictly have a probability sample any longer, but generally it is close enough, and one can use estimation techniques based on the selection probabilities.

Stated slightly differently, I think the point of the question was that you cannot use estimation techniques for inference that assume random selection, if you did not have random selection in the first place, and that is a very good point. I just wanted to say it is not the only way, and I obtained a very great deal of good use from model-based sampling and estimation, for many years, and that use has expanded, and may be expanding further in the near future.  

Inferential statistics, unlike descriptive statistics, is a study to apply the conclusions that have been obtained from one experimental study to more general populations. This means inferential statistics tries to answer questions about populations and samples that have never been tested in the given experiment.

Inferential statistics infer from the sample to the population, they determine the probability of characteristics of a population based on the characteristics of your sample, they also help assess the strength of the relationship between your independent variables, and your dependent (effect) variables. With inferential statistics, you can take the data from any samples and make generalizations about a population. 

There are two main areas in case of inferential statistics, estimating parameters. This means taking a statistic from your sample data and using it to say something about a population parameter and hypothesis tests, this is where you can use sample data to answer research questions. 

Regards