I am working on the sample size calculation. There are many outcomes and many independent variables needed to be tested. The type I error rate will be increased due to many hypothesis testings. For sample size calculation, is it needed to consider the inflated type I error?
Moreover, can a larger sample size copes with the inflated type I error?
thank you
I agree with what others have stated. Increasing sample size will reduce type II error and increase power but will not affect type I error which is fixed apriori in frequentist statistics. In the case of multiple outcomes and variables, if you want to test them simultaneously then you need to adjust for type I error. There are many methods but Tukey's method has more power than other methods like that of Bonferroni, Scheffe, Duncan, Fisher's LSD etc. Bayesian informative hypothesis testing is more flexible than the frequentist methods discussed above. Type I error is not fixed apriori here. However you need to set up its prior distribution on the basis of which it posterior distribution can be estimated.
As pointed out by Robert, “It's always a trade-off between alpha and beta errors. Of course, larger sample sizes make many things easier. But given, that you assign your Type 1 error yourself, larger sample size shouldn't help there directly I think and the larger sample size only will increase your power.” True the larger your sample size, the surer you can be that it truly reflects the population. But question arises how large size of a sample should be. Users of the following formula to determine an optimum sample size have generally reported satisfactory results:
Sample size (n) = [Chi-square*N*P*((1-P)]/[(E-square*(N-1)+(Chi-square*P*(1-P)].
Chi-square is table value at 1 degree of freedom for the desired level of confidence (0.05; 0.025; 0.001 etc.),
N= total population from which sample is to be randomly taken,
P= population proportion, since one cannot be sure what proportion of population should be in a sample, generally it is kept 50 % or 0.5. However, .4 or .6 may also be tried.
E-square= square of desired Margin of Error as specified in the case of Chi-square at one degree of freedom.
Download original source from the link:
http://opa.uprrp.edu/InvInsDocs/KrejcieandMorgan.pdf
The larger sample size could reduce the overtraining probability according to the Vapnik's theorem. This means the probability that precise value of error differs from estimated on some small value P(|P(q)-P(q^)|etha on the basis of training set (for example). But according to this theorem the sample size should be 10^5 or 10^6 to have small value of overtraining probability. This is correct but useless in practice. To use this in practice some others estimates were built taking into account the profile of concrete algorithm
Homoscedastic methods (equal variance assumption) use the wrong standard error when indeed there is heteroscedasticity. There are papers showing that as a result, they are not asymptotically correct. That is, large sample sized do not necessarily save them. Testing the assumption of homoscedasticity is not supported by six published papers. In practical terms, always use a heteroscedastic method. Things improve more using modern robust method developed during the last quarter of a century. As many tests, there is a vast literature on this with quick explanations seemingly impossible. Suggest looking at
Wilcox, R. R. (2012). Modern Statistics for the Social and Behavioral Sciences: A Practical Introduction New York: Chapman \& Hall/CRC press
or
Wilcox, R. R. (2012). {\em Introduction to Robust Estimation and
Hypothesis Testing 3rd Edition. San Diego, CA: Academic Press.
for recent summaries of the many methods that might be used.
(Will be giving an invited Talk at your university in Dec. Stop by if you still have questions.)
large sample size doesnt control type I error rates.In caluculating sample size of the study there are several ways one can adjust for the Family wise error rate(FWE).The easiest one is apply bonferroni correction in the caluculation of sample size instead of Z alpha we take Z alpha/no of comparisons.There are other methods also.I am attaching a file which will guide you to choose write method.Group sequentials and adaptive designs are feasible if study is a clinical trial.Also there are pratical issues in implementing these designs.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics