I made the mistake of not factoring in that crowdsourced samples from Prolific (similar to MTurk) are by their very nature convenient, and so they aren't generalisable. The survey was intended to collect a representative sample.
To give a little info on the survey: sample size is 400. It's a precursor to a series of experiments. It includes repeated measures with the intention of identifying relationship strength for respondents to two types of groups. The strongest respondent-group relationship would then feature in the experiments. What tests can I run on crowdsourced data so that I can generalise the results to inform my future experimental methods? Am I able to use parametric tests on the data or will I be limited to non-parametric?
Hi Rose
In order to select a suitable/convenience sample, the following steps/principles have to be followed:
1. Research studies are usually carried out on sample of subjects rather than whole populations. The most challenging aspect of fieldwork is drawing a random sample from the target population to which the results of the study would be generalized.
2. The key to a good sample is that it has to be typical of the population from which it is drawn. When the information from a sample is not typical of that in the population in a systematic way, we say that error has occurred. In actual practice, the task is so difficult that several types of errors, i.e. sampling error, non-sampling error, Response error, Processing error,… In addition, the most important error is the Sampling error, which is statistically defined as the error caused by observing a sample instead of the whole population.
3. The underlying principle that must be followed if we are to have any hope of making inferences from a sample to a population is that the sample be representative of that population. A key way of achieving this is through the use of “randomization”.
4. There several types of random samples, Some of which are: Simple Random Sampling, Stratified Random Sampling, Double-stage Random Sampling... Moreover, the most important sample is the simple random sample which is a sample selected in such a way that every possible sample of the same size is equally likely to be chosen. In order to reduce the sampling error, the simple random sample technique and a large sample size have to be developed.
5. The following factors are highly affected the sample size and need to be identified:
- Population Size,
- Margin of Error,
- Confidence Level (level of significance) and
- Standard of Deviation.
6. Then, the sample size can be estimated by,
Necessary Sample Size = (z-score or t-value)2 * StdDev*(1-StdDev) / (margin of error)2 .
7. For more details, please refer to
Cochran,W.G. (1977). Sampling techniques, Wiley.
Snedecor, W.G. and Cochran,W.G. (1989). Statistical Methods, Blackwell.
Regards,
Zuhair
Forget it. There are essentially no such procedures. If you want to make a less costly guess use a convenience sample. I would not trust an experimental design based on that. If the original sample gave "bad" results, the experiment will likely be poorly designed. See any good books on Sampling Theory and Experimental design. Best, David Booth
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size