I performed the Kruskal-Wallis test in R on 6 groups. I then proceeded to do a post-hoc Scheffe test. There is one pair (let's call them group A & group B) where the Scheffe test shows a significant difference. I know for sure that there isn't a significant difference because these are identical. As to why would I compare two identical groups will require a long explanation of the experimental design, so please allow me to skip that part.
I then went on to perform a Kruskal-Wallis test on only group A & B in R, which i assume essentially is the Mann-Whitney U test since there are only two groups. When comparing only these two groups, the results show that there is no significant difference, which is what is expected!
May I ask why is this happening?
What actually is the Scheffe test - does it perform the Mann-Whitney on each pairs, or is it something else?
I am not a statistician and this is a fundamental question. Just trying to figure out what is going on here. Please enlighten me, thank you.
When looking at statistical tests for groups. You need to consider are the data from a normally distributed variable (such as a continuous variable of height or weight.
A Kruskal Wallis is a non parametric test of groups where the data is ranks.
What we tend to do is to run the analysis , we check the results. Only if there are significant differences between the groups do we actually perform a comparison to determine any differences among the groups. Scheffe is usually a multiple comparison test performed on the means from an ANOVA. There are others and it depends on your questions about the groups which one you sue
Most basic statistics books will help with simple experimental design. It is more powerful and a lot easier to understand than the Kruskal–Wallis test, so unless you have a true ranked variable, you should use ANOVA.
Scheffe test would not be a usual follow-up to a Kruskal-Wallis test.
The recommended post-hoc for K-W is Dunn test. If you are using R, use the dunnTest function in the FSA package, as the functions in some other packages have some less desirable qualities. (See Details here: https://www.rdocumentation.org/packages/FSA/versions/0.8.16/topics/dunnTest ).
There's a worked example of K-W with post-hoc here: http://rcompanion.org/handbook/F_08.html .
As to why your Scheffe test is giving odd results, there could be something wrong with the function or something wrong with the way you are passing the data to the function. You would have to share a reproducible example for anyone to be able to determine the problem.
I am new to KW test. My problem is as follows:
I have data on performance of 60 administration units ( it is a survey data and there are responses from 30-40 respondents for each administration unit). I wish to inquire how statistically significantly difference these units are, and by how much. So I may expect results where 4-5 administration units are equally good (no statistically significant difference), while others are relatively weaker in their performance (again it could be groups). Hence, there may not be any strict ranking of these units (however strict ranking is what is required), instead a several clusters which are ranked by their statistically significantly different magnitude. Please guide.
Thank you Salvatore...
Yes, I have followed your advice and tried the Dunn's test. Everything is well now. However, I am still curious about the results from using the Scheffe:-
1. Why is group A & B significantly different when I performed the test for all 6 groups, but not significantly differently different when I only tested for groups A & B?
2. Second question - In Scheffe, what test is actually performed between the pairs?
Thank you also for your handbook. Very helpful and easy to follow!
Cheers...
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA