The accepted paper many times does not depend on good sampling. I have seem some periodics like PNAS to accept paper whose sampling design was clearly insufficient to robust outcomes. Many times the most important is the name of auhors than anything else.

As Joe Mudge stated, a biased sampling procedure can lead to unjustified conclusions. That being said, we are in many cases forced to make non-random sampling due to external conditions.

To increase chances of your paper being accepted despite a non-random sampling, you could try to justify why you sampled by one method (i.e. non-random) instead of another, that would have been obvious (i.e. the random).

When you have achieved some results, you can increase the power by e.g. bootstrapping or other sampling methods.

Finally, remember to state that your results are conditional on the initial biased sampling procedure.

Thanks all, I agree with you that samples taken and size should be discussed with statisticains before starting collection. But, the problem is that someone carried out the work after taken the samples and analysed in selective way and most of the work was done as well as writting the manuscript but the statistician insistted that the samples were not taken proparelly. Shall the paper to be sent or not.

I don't think justifying "why" the sampling was non-random is enough unless it is also possible to quantify and correct the bias that is transferred from the non-random sampling design to the statistical results. Sampling method determines the appropriateness of the statistical analysis, and when ignored can lead to grossly erroneous results.

A transparent presentation of the non-randomness followed by a coherent and reliable analysis that corrects for the bias or quantifies the uncertainty generated by this bias is necessary for publication. If the bias is unavoidable, then specific statistical sampling designs need to be developed that include necessary auxiliary data to quantify and correct for the bias. The statistical sampling literature is full of examples including size biased sampling, cluster sampling, sampling with detection bias, non-response error etc.

In general:

Precision of estimation can be improved through increased sample sizes, but only correct and representative sampling can assure accuracy.

In this case your study will be just exploratory or pilot study, you cann't do generalization or draw inference about your target group

Hello everybody,

I will also agree with the aforementioned comments and suggestions. In my opinion since there are doubts about the way that data was collected you need to correct any bias and justify carefully all the data collection procedure etc before sending the article for publication.

It is better to answer all the potential questions rising from your research and methodology before submission rather than being asked tonnes of questions from the reviewers!

Good luck

Hello everybody,

The random assignment is an important step to avoid a false conclusion. Even if you have a justification in your sampling, the results remain not reliable in statistical point of view. However there are a statistical methods which allow to reduce this bias as for example the method base on the propensity score.

Random assignment is generally used in experimental and quasi-experimental designs. The use of a nonprobability sampling design very much depends on your research question and research design. For example, qualitative research (such as a focus group) rarely if ever utilizes probability sampling designs. Qualitative research or exploratory research focuses on topics that have received no or liffle knowledge. Explanatory research, on the other hand, almost always requires probability sampling designs.

It's not really as simple as whether a specific choice weakens likelihood of acceptance. Rather, it's a question of /why/ you made that choice, and how clearly it is presented in the article.

For example, an article that stated clearly that it uses pre-experimental methodology due to X and Y limitations of the setting, but nonetheless found something incredibly interesting and worth sharing, could certainly be published - but one that just tried to gloss over badly crafted method, not a chance.

And so - if non-random assignment was a reasonable choice, and the rest of the paper reflects its implications, submit. If it was just shoddy work, don't bother. :D

In some fields, such as palaeobiology, you usually only have got imperfect samples (incompleteness of the fossil/ geological record).

Especially for palaeo(bio)diversity estimates some methods to reduce biasing through sampling effects have been applied, e.g. rarefaction to counter sample size differences, or correction by means of independent geological data (e.g. chance of a fossil to preserve and to be collected is assessed for a certain area, time slice or type of rock).

Effects of imperfect sampling are usually expected to be discussed by the authors but, given that they are more or less unavoidable, they not necessarily stop a paper from being published.

If you can justify why you opted for non random sampling that you can accept the findings of the research as an intermediate results which you can triangulate at the end by a focus group exercise or any other legitimate interviewing method to test and confirm your earlier findings. Again, all this depends on your research questions and research design. Good luck.

As far as sampling design is concerned it depends on the problem under study. But in general random sampling is preferred over non random i.e. subjective/purposive sampling because it gives a true representation of the population without bias. But this is not the case that non random sampling is always gives wrong or faulty conclusions. If your problem seeks subjective sampling then nothing wrong in doing that but you should verify it once more. And as far as rejection or acceptance of a manuscript on the basis of sampling design is concerned it merely depends on the problem's behavior. let me explore this one suppose one has to conduct a survey to estimate the average per capita income then if he will collect a subjective sample (say he collects the data only from the rich localities due to some personal bias) then it will not reflect the true picture and lead wrong conclusions. like this selection of sampling design depends on the problem itself.....good luck.

In (bio)medical science there is rarely random sampling. If some animal model is used, the animales are taken completely by feasability and not at random - with respect to the degeree of generalization done from such experiments. Results are reported as being generalizeable to e.g. "mice" or "rats" although not all mice or rats on this world had the same chance to get into the sample. However, for a given feasbility-sample, the assigment of treated and control animals is often done quite randomly. The practice is worst when it comes to patient data. Here we do have feasability-samples (patients have to be known, they have to be at the right time in the right clinic, and have to be willing to participate in a study etc.). Not each patient meeting the inclusion criteria in the world, not even in the country or the city has the same chance to be part of the sample. And further, here are many important covariates associated with the location/culture/economy/social status/environment... that are usually not considered, although generalizations are made far beyond the population of patients that had equal chances to get into the study.

So from this point of view I would say that random sampling is not common in (bio)medical sciences (but random assignment is, what somewhat reduces bias - but not at all completely) and results of such essentially non-random samples are routinely published. This might be one of the reasons for the dramatically high false-positive rates (see eg PLOS Medicine 2005 vol 2 Issue 8, e124, or NS Young et al PLOS Medicine Oct. 2008, Am Soc Clin Oncol Educ Book.2013:466-8, and Nature 483, 531–533 (29 March 2012).

Jochen,

I'm in agreement with your discussion of the issue. I think it would help if authors made clear the limits of statistical inference to the population available for sample inclusion in contrast to the full population to which statistical inferences are often extrapolated deductively,