I conducted a chi square test using crosstabs i.e. (2x3 matrix of Gender x frequency of purchase) and found that there is a significant difference (p=0.037). Further I conducted a post hoc test using standard residual method suggested by Beasley and Schumacker (1995). I got following adjusted residual values -2.3, 0.3, 2.3, 2.3,-0.3,-2.3 (which seems good since greater than 2.00 is rule of thumb). Standardized residuals have been appropriately adjusted for the Type I error rate. In this case, there are 6 cell values being tested, resulting in an αadj of .0083. I further calculated p value and observed that, the standard residual of no cell is significantly contributing to significant omnibus chi-square statistic. How to interpret this result?
Firstly, the chi²-test and the post-hoc tests test different hypotheses. Comparing their results is like comparing apples and peaches.
Secondly, p0.05 does not proof that there is no effect. So there is no factual contradiction at all.
Finally I wonder if and how you defined a "type I error". This error relates to a substantial alternative hypothesis, and I bet you haven't defined such a hypothesis and that you have not designed the experiment to be analyzed with a hypothesis test. Instead I assume you actually do a significance test - for which "type-I error" has no meaning, because this test is only about the probability of the data under H0, and H0 is a theoretical construct, a condition in a theoretical model, and there is no question if this is "true" or "false". It is just a kind of "reference point" or "leverage point" to get a p-value that may serve as an estimate of the "statistical signal-to-noise ratio" in your observed data. Since Type-I errors are undefined in such a setting I also don't see any particular logic in controlling a family-wise type-I error rate (however, adjusting the p-values may still be godd idea, but their use to "select candidates under control of the FWER" is non-sensical).
Firstly, the chi²-test and the post-hoc tests test different hypotheses. Comparing their results is like comparing apples and peaches.
Secondly, p0.05 does not proof that there is no effect. So there is no factual contradiction at all.
Finally I wonder if and how you defined a "type I error". This error relates to a substantial alternative hypothesis, and I bet you haven't defined such a hypothesis and that you have not designed the experiment to be analyzed with a hypothesis test. Instead I assume you actually do a significance test - for which "type-I error" has no meaning, because this test is only about the probability of the data under H0, and H0 is a theoretical construct, a condition in a theoretical model, and there is no question if this is "true" or "false". It is just a kind of "reference point" or "leverage point" to get a p-value that may serve as an estimate of the "statistical signal-to-noise ratio" in your observed data. Since Type-I errors are undefined in such a setting I also don't see any particular logic in controlling a family-wise type-I error rate (however, adjusting the p-values may still be godd idea, but their use to "select candidates under control of the FWER" is non-sensical).
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Applied Statistics