I have 14 treatments that i have analysed using ANOVA. There were no significant differences. Just by looking at the treatment means I am convinced there are differences. So which other statistical test can i use. I even used the Kruskal wallis test, since the data is not normal.
Trying out different tests just to find one that gives a result you prefer is known as "p-hacking" (also called "fishing", "research degrees of freedom", also related to the "garden of forking paths", etc.). See the readings linked below.
http://pss.sagepub.com/content/22/11/1359.full
http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdf
please try to increase the number of the observation of each treatment
Dear Betty
you must test your data for normal distribution, the normally distributed data can be analyzed by ANOVA , but in the case of non-normal distribution, you must use nonparametric test ( Kruskal-Wallis test ). The significance of the analysis affected by the variance within a group, then you notice a remarkable difference between means of groups but not reach to the significant level.
Good Luck
Trying out different tests just to find one that gives a result you prefer is known as "p-hacking" (also called "fishing", "research degrees of freedom", also related to the "garden of forking paths", etc.). See the readings linked below.
http://pss.sagepub.com/content/22/11/1359.full
http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdf
Thanks everyone, i have arcsine transfromed the data, then ran an ANOVA and i did get significant differences. Next time i will definitely increase the number of replications.
What was the motivation for arcsine transformation? If it didn't have any a priori motivation and the only reason for preferring it is that it gave p
In line with Stephen's comment, I would suggest in future that you have a robust analysis plan in place *before* collecting data, to avoid the kind of situation where you are running multiple tests on the same data - there is of course the risk of false positive when you do this.
You must test your data for the normal distribution, the normally distributed data can be analyzed by ANOVA.
Good job!
Why not thinking to use the repeat measure by SPSS. In this case you have to set the matrix in horizontal: 14 baseline and 14 first period, and also sort the the cases in such way the first is the same in the baseline and first period. Try it
- Badges
- Science topic