Hi Ekaterina,

That is a great question.  Here is a terrible answer, "it depends".  If I am asking questions in which the motivation to answer the survey is critical (i.e. people not interested in the topic will not answer) I would use a random survey.  To be honest, it might not be so "random" as there would be a significant amount of self-selection.  For example, you opt to survey saving habits, people with money in the bank might be more motivated to answer than those who are broke. 

On the other hand, if you are surveying for overall attitudes, random surveys will help your generalizability, depending on your response rate.  We routinely see political surveys in the US showing they "randomly" called 645 people, but what they do not tell us is how many hung up without answering.  I could make the argument that though the sample they got was random, it was a random selection of those with greater political motivations than those who simply refused to answer the phone or hung up.  

I guess the point is no matter how innocuous the question, some group of people will be highly motivated to answer it, so even within your randomization there is a degree of "quota-like" behavior.  

If not to sensitive, what question does your survey ask?

Best wishes,

Rich

Dear Rich, thank you for your answer. I share same opinion, but in practice most of the survey agencies use quota samples and then make general conclusions. Unfortunately we have example where a researcher spoke about 'random quota' and estimated random error. My aim with my article is to present the current practices of sampling, the real research motives and of course to address the issue with some of the more actual references. Do you know if this question has been explored in publications?

Best,

Ina

Hi Ina,

I found this:

http://www.aacp.org/meetingsandevents/pastmeetings/2012saee/documents/2009%20bestpractices%20for%20survey%20research.pdf

Hope it will help...look forward to seeing your paper.

Rich

Though random sampling in any would be more representative, many a times we do not have the records of full population and thus has limitations in implementation. In such cases, quota sampling after calculating an estimate of the sample size makes sense. If the population is diverse, then stratified quota sampling also practised.

Dear Ekaterina,

my opinion on that - as an academic researcher and a practitioner as well - is that in the end of the day, you are guided by the accessibility of your target population.

Random sampling - as you know - is based on the assumption that all members of the target population should have exactly the same chance of being selected. You may be able to conduct a random selection for a group of University students or a group of regional companies...But most of the time, it is almost impossible for a very big population, such as a city or a country. 

Therefore, researchers facing those latter situations may have no other solutions but to go for empirical sampling procedures. 

In theory, I agree with you that empirical methods cannot reach the level of reliability of random sampling and that - in theory again - most of the statistical tests performed later are not really applicable, if the probabilistic conditions are not respected...

Best regards

Stephane

For a simple random sample, every unit of the population has the same probability of selection, but there are random sampling methods where this is not the case or even desirable.

Non-probability sampling, including quota sampling, is model dependent.  The model must be appropriate and the statistician must be able to defend the model.

Thank you very much for your comments.

Dear Stephane, the quota samples in Bulgaria are used instead of random ones in population studies - for electoral surveys for example. One reason for such research choice is because we have no public access to population registers, so address samples could be done with random walk.

You all helped me a lot, thank you once again!

Ina

Population registers are not available in the U.S. either. That was one reason for the switch some years ago to an emphasis on "random digit dialing" via telephone.

Random sample does not mean that it has to be simple random sample. Complex random samples are used in most of surveys: stratification by region, cluster of sensitarian areas or blocks then a search for household and etc. You don´t need to have a population register for that. Complex samples are random samples.

Dear David, yes, I agree. That is why the professional community in Bulgaria is also trying to find the best alternative, but for the current moment in my country the 'random digit dialing' via phones cannot be used for representative surveys.

Dear Maria, complex random samples are in use in Bulgaria in various forms. thanks for your remark.

Quota sample does not guarantee random sampling, and as a consequence we do not know about its precision or unbiasedness of estimators.