What is the general consensus on one vs. two-tailed hypothesis testing in planned contrasts? I have a repeated measures mixed design. The study consists of three groups (A, B, C) and we have three assessment comparisons, i.e. Time 1 vs Time 2, Time 1 vs. Time 3 and Time 1 vs. Time 4.
Group A is our intervention group
Group B is an active control group
Group C is a control group
Our hypotheses are directional:
Group A > Group B
Group A > Group C
Group B > Group C
In this case, would applying the one-tailed significance test be ok?
Best
Martin
Based on your hypotheses, the one-tailed significance test is okay. If you consider Group i not equal Group j (for all i not equal j), then you need to consider a two-tailed test.
Md. Belal Hossain , thank you. I suppose that you refer to unequal samples sizes above? Group A, n = 31, Group B, n = 28, Group C, n = 31.
Good morning Martin Jensen. IMO, you should only use one-tailed tests if it is sensible and defensible (from some theoretical point of view) to treat large differences in the "wrong" (i.e., unexpected) direction exactly the same way you would treat differences = 0.
What is your research question? It appears you have the hypothesis
A>B>C
Is your research question just about the three comparisons individually, or do you have the single hypothesis above? Are you trying to estimate the size of these effects?
Daniel Wright , We expect A > B, A > C and B > C. Does this make sense? We base our hypothesis on the fact, that the treatment for intervention "A" is based on a neurophysiological model of tinnitus, whereas B is not. Previous studies that have used this treatment protocol have found it to be effective in reduction in tinnitus distress. However, this study is the first to compare "A" directly with something else, i.e. "B" and "C". We expect B to outperform C because of heightened placebo response in B.
The unequal sample size is not a problem in your case. The attached link is an excellent resource you may find reading interesting: https://www.bmj.com/about-bmj/resources-readers/publications/statistics-square-one/7-t-tests
So finding B > C should replicate past research, and the tons of information on placebo effects. Your interest is in A v B, where A does something in addition to B. As I see it, this one is your hypothesis that presumably you did your power analysis to detect, not 3 comparisons (and planned v unplanned would still be 3). I would focus on this one. Following what Bruce Weaver said, since presumably it is not inconceivable that some unforeseen "thing" means in the population A < B, I'd do a two-tailed test, but reporting the effect size with some interval.
I'll expand on what Bruce said. You should only be testing a hypothesis if a reasonable person could believe that it is true (or almost true, it is a point so will always be false, but ...). If a reasonable person believes A = B is plausible, surely another reasonable person believes A < B. And if a reasonable person believes that then for most research questions a two-tailed test is preferred. There are situations where external forces mean you do only care about effects in one-direction, but those at least in my field (education) are rare. Be interesting if other people agree with me on this.
Daniel Wright Bruce Weaver Md. Belal Hossain , thank you all for taking time to advice and guide. Much appreciated. Bottom line. If in a room with 100 reasonable people, some believe B > A or even B = A, then two-tailed as I understand the above. But would that not depend on the scientific rationale or justification? I mean, what if a model of the world supports or favors a particular hypothesis, would it then not be reasonable to use one-tailed tests? I guess this is also a philosophical issue and to me, seems rather complex. Again, I have seen others express the opinion that one-tailed is not carried out enough and that we miss valuable insights because of it. What it all boils down to, I guess, is how it is received by the peer-reviewers as well. I am quite sure a "one-tailed" approach will be rejected. However, I thank you all for your contribution to this. Not sitting back with a 100% conviction of how to approach, but I will figure it out.
Let me put it this way: If you plan to use one-tailed tests for A-B, A-C and B-C (with the expectation that all of those differences will be positive), if any difference is negative and large enough to be statistically significant by a two-tailed test, you're stuck with treating that large difference in the "wrong" direction exactly the same as a zero difference. To do otherwise would be cheating.
I suggest that you use the formulation that tests what the experiment was designed to test. Sometimes that means as Dr Weaver suggested you might need to redo an experiment with a somewhat different design. Research is like that. Best, D. Booth
You can use one tail test...for A>B>C.you can make three hypothesis
1: H0: A>B
2: H0: B>C
3: H0: A>C
The concept of one-tailed/two-tailed tests makes sense only for A/B-tests (Neyman/Pearson), not for Fisherian significance tests, becauseonly then there is somesensible definition of type I/II errors that could be set to minimize the expected loss. So if you do significance tests (tests of H0, not A vs. B), the aim just to see if there is sufficient data to interpret a sample estimate relative to H0. This task, by definition, has no "side" or is, by definition, always "two-sided".
If you "expect A > B, A > C and B > C", what means: A > B > C, you could simply test the monotone association between the group (coded numerically so that xA > xB > xC) and the response.
- Badges
- Science topic
- Similar topics
- Biological Science
- Molecular Biology
- Biomolecules
- Nucleic Acids
- DNA