After ANOVA in SPSS, I used post hoc tests to determine which mean differences are significant and which are not. When I compare data among group 1,2,3,4, the results show that group 1 is significant different from group 2, 3, 4. There is no significant difference between group 2, 3, 4. However, when I compare data among group 2, 3,4, the results show that group 2 is significant different from group 3, 4. Then I am confused whether there is significant difference between 2, 3, 4. I have a hypothesis that the large data in group 1 might cause the problem decribed above. Can anyone point out how to perform significance analysis under this situation?
Hello Pieru,
If your intention is to run all possible pairwise tests (as it apparently seems to be), then recognize that:
1. The error term for the first set of pairwise tests will be quite different in the full, four-groups batch of data than in the second, three-groups batch. That's because the tremendous variance in group 1 is overpowering the pooled estimate. The second (groups 2-4 only) will have a smaller error term because the MS(within) will be considerably smaller. That alone will explain why what you're implying as being anomalous results have occurred.
2. Did you judge the full data set suitable for anova in the first place? Have a look at summary statistics:
group Mean N Variance
1.00 448927.0000 3 3017679364.000
2.00 7500.0000 3 1480000.000
3.00 143.3333 3 13633.333
4.00 2.3333 3 .333
Total 114143.1667 12 41315414697.424
Clearly, any suggestion that the respective subgroups are homogeneous with respect to score variance is absurd (and the data set sure looks like a professor-made course example or problem).
Other approaches would be preferred here (besides straight anova). There are variance-adjusted anova options, there are exact/permutation and resampling/bootstrap options, and so on.
Professor Booth's response is an attempt to get you to look at methods which can adjust for unequal variances in follow-up ("post hoc") testing.
Good luck with your work.
It really looks like a made-up example. It is so very clear that at least the assumption of similar variances in the groups is drastically violated. The ANOVA therefore severy lack power with such tiny sample sizes. I suppose that the actual task was to recognize this and to identify a reasonable distribution model rather than blindly applying an unsuited tool.
Ordinarily, the quantity and quality of data used in statistical analysis are related to the accuracy of the output
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size