Is there some methodology or reasoning that we can follow for this selection? I understand the difference that for two-tailed are of the tail is equally distributed on both ends.
But is there any need to select one over the other? How to decide if I encounter new data, what test to go with?
The question if you should use a one- or two-tailed tests makes sense only in the context of A/B testing (Neyman/Pearson). Even there it's not a question but follows from the way how you define your alternative hypothesis.
If you do significance testing (Fisher; calculate a p-value under a H0), the aim is to see if the information in the data is sufficient to reject H0, and thus to interpret the direction of the observed effect. This is always and neccesarily is a "two-sided question". Power and sample size, as Ohanes Haroutoun Ashekyan mentioned, are not a concern to decide between one- or two-sided tests. You get a p-value and you have to judge if it is small enough to reject H0. If you think that, in the context of your research, p < 0.1 is sufficient, than go with it. If you think that p < 0.02 is not yet sufficient, than go with that. If you fail to reject H0 because p is still larger than your critical value, then your conclusion is: "get more data" or, if this is not possible: "think of a different kind of experiment providing more information about the parameter of interest", and if this is also not possible: "state that your data is not conclusive regarding H0" and note that it may not be feasable in future to repreat such experiments.
The combination of p-values (significance tests) and one-/two-sided tests (that are actually defined only in the context of A/B testing) follows from the unfortunate and incoherent and fawlty NHST mixture approach that is described in most introductory stats books.
Ashay Singh
There are a few considerations to make:
if you are able to previously predict your outcome i.e. you expect to see an increase or a decrease, then you should use a one-sided alternative hypothesis. However, if you cannot anticipate the direction of your result, then a two-sided alternative is more appropriate, where it tells you if there is any significant association between your tested groups, but cannot tell you about an increase or a decrease i.e. cannot give you a direction.
Another consideration is to at the level of the sample size: if you have a small sample size, it's better to go with a one-sided alternative. However, large sample-sized experiments are suitable for the two-sided alternative.
The last consideration would your decision of how strict you want your test to be. if you want a more strict i.e. a difficult-to-achieve statistical significance i.e. when you reach it, this means that your result is so significant, then use a two-sided alternative i.e. don't divide your p-value by 2. And if you want to conduct a less strict i.e. an easier to reach statistical significance, then use a one-sided alternative i.e. divide your p-value by 2.
I hope you benefit from the comment,
Best
The question if you should use a one- or two-tailed tests makes sense only in the context of A/B testing (Neyman/Pearson). Even there it's not a question but follows from the way how you define your alternative hypothesis.
If you do significance testing (Fisher; calculate a p-value under a H0), the aim is to see if the information in the data is sufficient to reject H0, and thus to interpret the direction of the observed effect. This is always and neccesarily is a "two-sided question". Power and sample size, as Ohanes Haroutoun Ashekyan mentioned, are not a concern to decide between one- or two-sided tests. You get a p-value and you have to judge if it is small enough to reject H0. If you think that, in the context of your research, p < 0.1 is sufficient, than go with it. If you think that p < 0.02 is not yet sufficient, than go with that. If you fail to reject H0 because p is still larger than your critical value, then your conclusion is: "get more data" or, if this is not possible: "think of a different kind of experiment providing more information about the parameter of interest", and if this is also not possible: "state that your data is not conclusive regarding H0" and note that it may not be feasable in future to repreat such experiments.
The combination of p-values (significance tests) and one-/two-sided tests (that are actually defined only in the context of A/B testing) follows from the unfortunate and incoherent and fawlty NHST mixture approach that is described in most introductory stats books.
One or two-tailed hypothesis depends on the direction of the question outcome. For example, if you are interested in testing a drug and you know that the drug has more side effects than the control, you should use a one-sided test. In other cases, if you wanted to test both (greater or lesser) outcome direction, the two-tailed test is more appropriate. Read detail here https://www.reneshbedre.com/blog/hypothesis-testing.html
- Badges
- Science topic