Hello Researchers
I am wondering that under what conditions it is more suitable to use an alpha value of 1% instead of 5% (using t-test). I have some data. When I use alpha value of 5%, every group is significantly different from almost every other group, but it is not significantly different while using an alpha value of 1%. I am not well educated in statistics, so any help would be greatly acknowledged. Thank you
Hi,
What is more important is the confidence intervals (C.I), whether they are wide or narrow. If the C. It is narrow with a given alpha value then it is sufficient, otherwise go for a smaller alpha value. Six sigma quality in some industries is required where still lower alpha values are used.
If you want to compare more than two groups, it would be better to use an F-test than conducting multiple t-tests.
Ultimately, the value chosen for alpha is arbitrary. Conventionally, however, a rejection level of .05 is considered sufficient to reject the null hypothesis. A researcher might adjust (decrease) the alpha value if they are conducting multiple hypothesis tests with the data and want to decrease the chance of family-wise errors. For instance, by using a Bonferroni correction.
A researcher might also set alpha to .01 if, given the nature of their study, they believe a false positive is practically more problematic than a false negative.
As Maik said above, it is quite arbitrary. However, there are two things that you could consider to decide your alpha%. First, the field of research. As said above, an alpha of 5% will increase the probability to reject your null hypothesis, but will also increase the false positive rate (the probability to find effects by chance although they are not real). In humanities and the social sciences, it might be quite hard to find positive effects as variables are usually analysed at a behavioural level. This implies subjectivity, psychology, emotions...that famous "black box" that is almost impossible to analyse. However, in some fields of research that are at the molecular level such as pharmacology or physics, an alpha of 5% (which assumes 5% of chance to make mistakes) would be a disaster. Can you imagine a research in nuclear safety that accepts 5% of error ^_^? Hence, the more subjective your field of research, the more lenient you can be. The last thing is how new is your topic. If you lead an explanatory research, then you can decide to accept a 5% alpha, at least to find interesting trends for future studies.
Hope it will help !!
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size