Hi!
I am facing a dilemma in reporting the planned contrast results for my new manuscript. I have two one-way ANOVAs each with three groups (say the first ANOVA has group A, B, C and the second has group D, E, F) I only want to compare A with B, A with C; and D with E, D with F on all DVs that I aim to examine. They are based on literature. However, there are several planned contrasts that show significant results (far less than 0.05), while the omnibus F is insignificant (far more above 0.05). I have found several references saying that if a planned contrast is significant then it is significant regardless the omnibus F (e.g., Chen et al, 2018; Howell, 2012 and webpage like this https://stats.stackexchange.com/questions/28845/what-if-an-overall-anova-is-not-significant-but-specific-contrasts-are). However, I tried hard on searching papers that used ANOVA planned contrasts and so far there is no paper reporting significant planned contrasts with the omnibus F being insignificant.
My question/dilemma is: 1) shall I only present the results of planned contrasts without reporting the omnibus F of ANOVA? or 2) reporting the omnibus F of ANOVA and arguing that the planned contrasts are valid regardless of the omnibus value? It seems to me that in either case it's a bit anti-intuition and anti-tradition and I am afraid of being easily rejected by reviewers due to this contradiction. However, if I just stopped at the insignificant omnibus F value, I will lose a lot of useful results and subsequent discussions for my paper.
Hope someone could help or provide some papers which present similar cases as mine!! ToT
Thank you so much!!
Reference:
Chen, T. et al. (2018). Relationship between Omnibus and Post-hoc Tests: An Investigation of performance of the F test in ANOVA. Shanghai Archives of Psychiatry, 30,60-64.
Howell, D. C. (2012). Statistical Methods for Psychology. Wadsworthing Publishing. (p. 372-373)
If groups A, B and C have no influence on groups D, E and F, then you can have separate ANOVAS. But how would you know about interactions until you test for them? Usually it's best to have one big ANOVA that includes all factors to capture all interactions and to reduce Type 1 errors. I review papers and I would definitely notice :)
Does the omnibus test have any rationale? It tests a hypothesis so vague that I've never used one in a long career. What hypothesis says that there's "some kinda difference"?
Report only on tests that are grounded in your study hypothesis and linked to theory. Otherwise you end up having to explain away results that you didn't need.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA