Can I use the result from a pilot study to decide the minimum sample size required to compare the means using independent sample t-test ? Or any other way that can help me to determine the appropriate sample size? (I need reference too)
The journal I'm applying for required the use of equation to calculate the sample size.
I'm done with data collection and I already collected 77 responses from an online based survey and this is the final sample size I can get.
The outcomes (scores) are presented in (%)
I divided them into two groups (depending on different variables) to compare the means of their scores (%)
Each group n > 30
Pilot study result:
SD from the pilot test of 11 participants = 7
More information:
Significance level (α) = .05
P.S the population size in undefined
Hello Bushra,
A priori determination of sample size depends on:
1. The specific hypothesis being tested (and the associated statistical test).
2. The desired amount of statistical power (higher is better).
3. The desired alpha (Type I error risk level).
4. The smallest effect or difference that is noteworthy (worth detecting), should it actually exist in the population (this is your target effect size: ES).
It sounds to me as if:
#1 is already decided (comparison of two independent means via the t-test);
#3 is already decided (.05).
#2 and #4 have yet to be declared;
You've already collected your data and decided that N = 77 (maximum, assuming no missing or inappropriate data points in the set).
I don't see much value in bothering with power determination given that you've already collected the data; it's not going to inform your data collection strategy in any way.
If you are obliged to report a power analysis, the only choices I see would be:
1. Compute sample sizes needed to detect an ES equal to what you observed in your pilot study (you didn't say what that was) and report
2. Report the power that an N of 77 would yield to detect a "small," "medium," and "large" ES (you could use Cohen's guidelines for this: 0.2d, 0.5d, 0.8d, respectively).
The G*Power program is very helpful for these types of computations (https://www.psychologie.hhu.de/arbeitsgruppen/allgemeine-psychologie-und-arbeitspsychologie/gpower.html).
Finally, please note that if your DV is a percent or a proportion variable, it may not behave very well as a normally distributed variable.
Good luck with your work.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Mahweh, NJ: Erlbaum Publishers.
Hello Bushra,
A priori determination of sample size depends on:
1. The specific hypothesis being tested (and the associated statistical test).
2. The desired amount of statistical power (higher is better).
3. The desired alpha (Type I error risk level).
4. The smallest effect or difference that is noteworthy (worth detecting), should it actually exist in the population (this is your target effect size: ES).
It sounds to me as if:
#1 is already decided (comparison of two independent means via the t-test);
#3 is already decided (.05).
#2 and #4 have yet to be declared;
You've already collected your data and decided that N = 77 (maximum, assuming no missing or inappropriate data points in the set).
I don't see much value in bothering with power determination given that you've already collected the data; it's not going to inform your data collection strategy in any way.
If you are obliged to report a power analysis, the only choices I see would be:
1. Compute sample sizes needed to detect an ES equal to what you observed in your pilot study (you didn't say what that was) and report
2. Report the power that an N of 77 would yield to detect a "small," "medium," and "large" ES (you could use Cohen's guidelines for this: 0.2d, 0.5d, 0.8d, respectively).
The G*Power program is very helpful for these types of computations (https://www.psychologie.hhu.de/arbeitsgruppen/allgemeine-psychologie-und-arbeitspsychologie/gpower.html).
Finally, please note that if your DV is a percent or a proportion variable, it may not behave very well as a normally distributed variable.
Good luck with your work.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Mahweh, NJ: Erlbaum Publishers.
David should have a minimal effect value of interest or clinically important. Cohen's criteria are questionable as findings vary in different clinical areas due to the type of study, the measures used and the sampling of the data. Therefore, the relevant scientific literature should be reviewed to establish the parameters of the effect sizes.
Article Beyond small, medium, or large: points of consideration when...
https://academic.oup.com/innovateage/article/3/4/igz036/5560156
Dear Bushra Ahmad ,
Why are you trying to calculate an ideal sample size when you have already completed collecting your data?
Just to add one small comment to David Morse 's excellent answer: you could use a pilot study to obtain an effect size estimate but you should be mindful of its standard error and it is better to use this in conjunction with effect size estimates already published for the same issue. But this would then guide the size of your main study, not be calculated after it has finished.
David Morse , excellent answer, but I would not adhere to Cohen's definition of "small", "medium" and "large" relative(!) effects. These are very hard to interpret in practice (getting a useful interpretation) and they are not at all linked to any kind of relevance - but they are usually mis-interpreted that way! They were developed for a narrow field in phychological research and should not be unreflectedly or reflexively applied to entirely different fields of research.
Peter Samuels , I am sceptical using published data to estimate effect sizes. There is very likely a considerable bias (over-estimation). If published data are used, I would recommend to divide the published effects size by two and use this as a reasonable belief about the expected effect size. I would also double the published variance to be used in my samplie-size calculation, because the variance in published data will likely be under-estimated for the same reasons as the effect size will be over-estimated.
Dear Jochen Wilhelm , thank you for your helpful suggestions. The subject of publication bias is very interesting in itself.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size