Hi everyone,
I and my colleagues are trying to figure out the sample size for a study that would represent the general population. We performed an ANOVA-based Power Test to ensure statistical robustness and reliability of the experimental design. However, the overall population which the sample represents was not among the inputs required in the power test, i.e. the overall population was not considered for the sample size generation.
Given that the overall population is in millions, a mere sample size of 9 seems awfully low, even if it establishes statistical soundness of the experiment, we think the results obtained in our study would not represent the general population and may not be extrapolated to larger groups because of the small sample size.
What might be a proper methodology to calculate sample size in light of the population it represents?
Thank you!
The target population comes into sample size calculations for surveys, but not power calculations for hypothesis testing. If you have correctly done the power calculation then a required sample size of 9 means that your expected effect size is huge, which is unusual in population health studies, and the small sample size makes it difficult to generalize to the wider population. You are therefore justified in suggesting a much larger sample in order to ensure that study subjects are broadly representative of the target population - that suggestion is based on generalizability, not power.
Hi Mohamed,
If you have good background info for the composition of your target population (% gender/%race, etc.), then you can design your sampling frames to randomly sample in proportion to the greater population (2-stage sampling). In this case, technically, any number you draw that satisfies the proportional composition would be "representative".
Hope this may be of some help,
Matthew
As pointed out nicely by Adrian and Matthew, representativeness and power are two very different things.
Representativeness is an issue of construct validity, i.e. whether you are capturing the phenomenon of interest. It is addressed through the sampling procedure, and may have implications for sample size, particularly in a heterogeneous target population.
Power is an issue of statistical conclusion validity, i.e. whether your hypothesis tests are correct. It is addressed through setting the appropriate Type I and II error rates and expected effect sizes in advance in order to compute the required sample size. That is all based on the assumption of arriving at the true effect size when taking an infinite number of samples.
That being said, a sample size of 9 does sound incredibly small, even per group. I would re-check whether the power calculations are correct. A software like G*Power might help.
Many thanks everyone for your insightful answers. They really did help.
A small sample size as a few disadvantages that are all based on the fundamental property that the small sample size does a poor job of modeling the distribution of rare events (standard deviation), and that a single sample with a small sample size will provide a poor estimate of the true mean. If you take many small samples the average will be the population mean, but there is no such guarantee for a single sample. This is sampling error.
If the process is invariant (or nearly so) then a small sample size is warranted. Equipment calibration using a calibration curve is one case where replication can be low. If the r2 is not close to 1 you do the experiment over.
If this is to satisfy an ethical review board or needed to justify sample size for a grant, then I suggest G*Power (http://gpower.hhu.de/). You can get to the same thing using proc power in SAS, and I would guess that R has a package or two that would work equally well.
Note that the sample size calculations will often require information that you do not have and cannot get before the experiment. The sample size estimate is only as good as the numbers you supply. Also, the sample size estimate is usually a lower bound. You will need to add in a few replicated based on your guess as to how much missing data you will have.
I suggest looking at published literature to get an idea of the sample size other use. However, realize that they too have had to balance time and costs. Therefore their sample size while publishable might be a bit lower than ideal.
Hi Mohamed
It is a real problem when working with big populations. As someone has recommended you, try to find a characteristic that define a part of this population.
A tool that could be useful for you is Open Epi: http://openepi.com/SampleSize/SSCohort.htm
The best wishes.
It all depends on the study design and the type of sampling you are going to use. If the sampling is not probabilistic, it will never be representative even if you use a huge sample. If, as indicated above, you are going to stratify the sample, the size should be increased. An n=9 is impossible to be representative of a population of millions. I recommend you a free and very powerful instrument, the EPIDAT, in which, depending on the type of design and the error you admit, it will give you the n you must recruit. It has manuals in English.
https://extranet.sergas.es/EPIWB/EPIWB/SolicitudeEpidat.aspx?IdPaxina=62715&idv=4&lng=en
I downloaded EPIDAT. It was in Spanish? I looked at the files and could not figure out how to start the program and could not find an exe file. There was a read me, but I could not really read it. I went to delete it from my system and could not find it. Searching for epidat did not turn up any file.
The EPIDAT is not installed, it is downloaded into a folder and from there it is started. To delete it, the folder is deleted.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size