I am conducting an intervention Qusai experimental research study. I want to know is it required to have a pilot study sample size to be calculated by any proper method? or I can take as little as 10-12 for both the groups (Treatment group and control group; no treatment-- 5 vs 5).
If no previous literature for the effect size and other evidence is available, what can help me justify this sample for the pilot study?
Also if I am intending to take for main study total sample of 60 (per arm n=30) then 10% rule of thumb here will be 6 so is it going to justify pilot study sample?
Lastly please recommend statistical tests for this small sample of a pilot study.
Thank you,
Pilot studies are not useful in determining the sample size of your "real" study (as they will always be underpowered, and lead to uncertain estimates; i.e. same as the issues with running an underpowered study directly).
See the recent preprint by Daniel Lakens which provides a nice intro into how sample sizes should be justified, even if no prior research has been conducted on your effect/research question.
https://psyarxiv.com/9d3yf/
However, a few things to note: consider carefully what you take to be a worthwhile effect (e.g., smallest effect size of interest) and the results you actually care about. Many software to calculate power will provide a sample size estimate for the Omnibus effect (i.e. if you have 3 conditions, the sample size is estimated to find an overall interaction effect, but not adequate to investigate the individual pairwise comparisons [which will be underpowered]). This is why you need a program that lets you select this option (e.g., MorePower, or the Superpower R package).
I completely agree with Mircea Zloteanu that it is not the aim of pilot studies to investigate the research questions on a larger scale and thus sample sizes for pilot studies should be determined on the basis of effect sizes and expected effects.
However, this does not render pilot studies completely useless, in fact pilot studies can contribute greatly for incresing the power in the main study. Pilot studies do so by identifying possible sources of unsystematic variance in the data and by doing so systematic variance and hence effects can increase.
For example pilot studies can investigate whether the instructions are understood by everyone, whether all scales are reliable and valid, or if the procedure takes to long and participants answer hastely and erratic towards the end.
The sample size thus depends on the amount of peoblems you expect to encounter, one general advice would be to repeatedly test and improve versions of the main stud with a handfull (10) of people instead of launching a beta with 50 participants.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size