I need to determine the sample size for my experiment. In my experiment I will select a set of x projects, from a population of N projects. For the sample of x projects will apply a treatment to know the proportion of projects that pass in this treatment.
I used the function ES.h of the package pwr. This function calculate the Effect Size between two proportions. For p1 = 100% and p2 = 95%, we have:
h = ES.h(1, 0.95) = 0.4510268
I understand that this effect size informs the need to detect the distance between the hypothesis.
I'm not very secure in my interpretation, but I used this value to determine the sample size.
pwr.p.test(h=h, sig.level = 0.05, power = 0.8)
Determining the sample size to detect up to 5 points difference in the proportions:
n = 38.58352
When I increase the Effect Size h, the sample size decreases. This is my interpretation? What do you think? Am I right?
Yes, increasing the effect size decreases the sample size. It is more likely to "detect" the effect when the effect that is to be detected is larger.
On what basis did you decide that the chosen effect is relevant?
Just for curiosity: what is your cost function, that is: how did you determine the values a alpha (0.05) and power (0.8)?
Thank you for your answer!
"On what basis did you decide that the chosen effect is relevant?"
I have little knowledge about Effect Size. I read some articles that they was citing Cohen where It was recommended Effect Size medium (0.5). I read that:
"The effect size is the degree of distance between the null hypothesis and alternative hypothesis distributions."
So, I decided to consider for my initial experiment a medium effect size.
When the effect size is smaller the sample size grows. Leaving the experiment prevented by time and amount of testing.
Only was defined the confidence level (alpha) and power level (beta error) that it had acceptable levels. I know, I am not very good determination. You have a suggestion?
I'm still new to using the Effect Size and Power. Do you have any recommended reading?
Thank you very much.
http://www.statisticshell.com/docs/effectsizes.pdf (and refs therein)
http://rpsychologist.com/d3/NHST/
http://www.leeds.ac.uk/educol/documents/00002182.htm
87
Using 80% power is really not acceptable anymore, either scientifically or ethically. To have a guaranteed 20% chance of failing to find an effect that is iortant in real life is not an acceptable use of research resources and the time and effort of participants. Go for 90% at least.
But, Ronán, a higher power also comes at an increased risk of erroneosely rejecting H0 when the effect is not important (that's not the type-I error, because this refers to rejecting H0 when the effect is point-zero and this is controlled by alpha).
In reaserch it can be that the costs (in time and/or money) of a measurement is not negligible, and iI wonder if it could be resonable to take a bad chance (withing the budget) instead of not trying it at all.
What do you think?
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size