Some statisticians advocate for the use of effect sizes and confidence intervals (CIs) rather than p-values when reporting analysis results. The issue is that p-values provide clearer levels of significance than effect sizes and confidence intervals.
Joseph Kaswengi
In general, point estimates and confidence ranges, or p-values, should be supplied wherever feasible. The extent of the effect and the quality of the evidence should be described in plain words.
P-values are more straightforward than confidence intervals. It is possible to determine whether a value is larger or less than a previously defined limit. This enables a quick determination of whether a number is statistically significant or not.
While a P-value can tell the reader whether or not an impact occurs, it does not indicate the magnitude of the effect. Both substantive significance (effect size) and statistical significance (P-value) are important outcomes to provide when reporting and analyzing research.
Thus, confidence intervals can be used to infer statistical significance' (equivalent to p0.05) - but they also reflect the biggest and smallest effects that are likely, given the observed data. This is valuable additional information.
Hi Joseph,
it depends on your research goal. If you test hypotheses, a significance test is the most reasonable thing to do. If you want to estimate a parameter and communicate the uncertainty involved in the estimation, reporting confidence intervals is informative. Both of course can be (and most often will be) part of most of the research activities (i.e., you want to test a hypotheis AND use the estimate for judging the effect size). Whether the H0 has to be nil is a different question as significance test test against some fixed value and that can be anything. Note: The "null" in null hypothesis (testing) does not stem from "null" (nil or zero) but from "the hypothesis to be nullified".
See
Lakens, D. (2021). The practical alternative to the p value is the correctly used p value. Perspectives on psychological science, 16(3), 639-648.
Best,
Holger
Confidence intervals around an effect size (e.g., correlation or mean difference) contain the information about statistical significance as well. If, for example, the 95% CI around a correlation coefficient contains zero, the correlation is not significant at the .05 level. In other words, if you report the effect size along with its CI, the p value becomes redundant/obsolete for testing the null hypothesis.
- Badges
- Science topic
- Similar topics
- Organizational Psychology
- Training