I have encountered a statistical test with very large type-I errors but also has a good power function at alternatives, Does such a test has any practical utility?
Please refer to the article at
http://www.phil.vt.edu/dmayo/personal_website/Berger%20Could%20Fisher%20Jeffreys%20and%20Neyman%20have%20agreed%20on%20testing%20with%20Commentary.pdf
Type I and type errors are inversely proportional to each other. The choice of test depends on the seriousness of type of error. If type 1 error is more serious then the test you are referring to isn't useful. If it's not serious then your test is better than other tests with power less than that of your.
I can't think of any practical utility for such a biased test.
Consider a simple extreme decision rule: :Always reject".
This decision rule has high power for any alternative (=100%) but the Type 1 error rate is also 100%. In general, power is only useful when the level of the test is controlled
DR Bristol gives a nice counterexample. If I may add consider usage of confidence intervals (CIs) instead of a test. CIs tell you not only whether there is an effect (with uncertainty of types 1 or 2), but also how large that effect is.
Cheers, Manfred
P.s.: If you need references, you may wish to consult my book on climate time series analysis; Ch. 3 is on CIs; the reference list is available in the free sample part of the book.
http://www.manfredmudelsee.com/book/index.htm
Type-1 error is the error of incorrectly rejecting the null hypothesis while the power of the test is the probability of correctly rejecting it. They cannot therefore be both high at the same time. I expect an inverse relationship between them. That is, when one is on the increase, the other one should be on the decrease. Hemangi, please I will like to see such a test.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics