Can somebody please help me? I conducted a research in which I used 15 questionnaire items to determine self-control among students classified into 5 groups according to their course of study. In analyzing the data I used the analysis of variance to compare the mean scores of the five groups on each item. Some of the items yielded significant p while others did not. The paper is under review and the reviewer recommended that I use the adjusted p value to reduce type I error. I therefore used the post hoc analysis with the Bonferroni method of correction in the SPSS to obtain the adjusted p value. The print out showed a pair wise comparison of the groups on all the items. I am now confused on how to report both the P value from the analysis of variance and the adjusted p value from Bonferroni.
Adjusting makes sense only if you want to control the family-wise error rate (FWER).You simply call these comparisons "significant" for which the null hypothesis can be rejected while keeping the desired (FWER). If you want to give the list/table of all comparisons with p-values, you can use one column to show with p-values and another to show the (Bonferroni-)adjusted p-values.
Note 1: Tukey's HSD is usually superior to Bonferroni.
Note 2: You seem to analyze "items" that are not measured on an interval scale. It thus can be that the "analysis of variance" as a tool for the analysis of interval-scaled (and approximately normal-distributed) data is inappropriate. Ok, the reviewer didn't seem to be concerned and so it will be unliekly that you will reconsider the fundamental analysis of your data... but you may possibly be warned for the next paper ;)
Adjusting makes sense only if you want to control the family-wise error rate (FWER).You simply call these comparisons "significant" for which the null hypothesis can be rejected while keeping the desired (FWER). If you want to give the list/table of all comparisons with p-values, you can use one column to show with p-values and another to show the (Bonferroni-)adjusted p-values.
Note 1: Tukey's HSD is usually superior to Bonferroni.
Note 2: You seem to analyze "items" that are not measured on an interval scale. It thus can be that the "analysis of variance" as a tool for the analysis of interval-scaled (and approximately normal-distributed) data is inappropriate. Ok, the reviewer didn't seem to be concerned and so it will be unliekly that you will reconsider the fundamental analysis of your data... but you may possibly be warned for the next paper ;)
Thanks so much Jochen Wilhem. Your answer has been so helpful. The data for the study was actually on an interval scale. From your explanation, I think I do not need any adjustment because there was no family-wise error rate (FWER). I only analysed the hypothesis on differences in the mean scores of students from different course of study. No other hypothesis was added to this to create the FWER
If you have 5 different groups and you want to identify differences between all possible pairwise comparisons using hypothesis tests, then you do have a family of 4+3+2+1 = 10 hypotheses/tests. Wheter or not is is reasonable to control the FWER among this family is a different question.
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software
- SPSS