In sample size estimation, before starting our research, we have to decide on the significance level and our estimated power for the study. When we are able to collect a larger sample size, could we consider as an example 0.01 instead of 0.05? How does this change affect the result?
Yes we can choose alpha level whatever we want. But the higher the alpha level (less than 0.05) the higher the cost of sampling. This can lower the power of your test result and higher the type II error if you consider
Yes we can choose alpha level whatever we want. But the higher the alpha level (less than 0.05) the higher the cost of sampling. This can lower the power of your test result and higher the type II error if you consider
You did not mention why you would want to lower your type-I error rate threshold, so I'm not sure if the following applies to your study. Regardless, I will share my thoughts......There are some good papers in the ecological literature that address methods for determining appropriate type-I and type-II error rates for monitoring programs based on the costs of each mistake. These methods are based on the idea that the ratio of cost of a false positive to false negative should determine the appropriate ratio of the type-I and type-II error rates. Once you know the appropriate ratio for your problem, you can adjust sample size to achieve the desired level of both. Hope this is helpful.
Joe Mudge, Leanne Baker, Chris Edge, Jeff Houlahan. 2012. Setting an Optimal α That Minimizes Errors in Null Hypothesis Significance Tests. PLoS ONE 7(2)
Field SA, Tyre AJ, Jonzen N (2004) Minimizing the cost of environmental management decisions by optimizing statistical thresholds. Ecol Lett 7:669–675
The type one error and the sample size are not directly related, so if you decide how often you are comfortable rejecting the null hypothesis when it is true, you should keep that no matter how large a sample you collect, especially if you already determined you can collect a large sample. A larger sample will increase the power of the test by effectively increasing the seperation between the distributions of the null hypothesis and the alternate hypothesis, and make a type ii error less likely.
Yes, you can choose any alpha level. Inaddition, if the alpha is larger than the prespecified alpha value, the power of test is also higher than the given power.
Mehdi -
You are on the right course. A p-value is a function of sample size, so balancing type I and type II error probabilities is somewhat analogous to a confidence interval around a point estimate with continuous data. If you can use a confidence interval, your result will be more practically interpretable.
The attachments may be of interest.
Cheers - Jim
Article Practical Interpretation of Hypothesis Tests - letter to the...
Conference Paper On the Lehmann power analysis for the Wilcoxon rank sum test
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size