Hello,
I have a question relating the p-value and confidence intervals from an ANOVA. The p-value, resulting from an ANOVA is an easy summary of statistical differences between more than two groups. However, I am not the biggest fan of p-values due to the fact they are not an effect size measurements, and, dealing with a large number of samples the p-value is always low. Is there any possible way to mention confidence intervals of an ANOVA in stead of p = 0.00356. I could mention D = 89 (Lower confidence interval = 67; Higher confidence interval = 102). Below I give an example of an R syntax.
df1
It sounds like you are mainly concerned with compressing the presentation of the results, while reporting confidence bounds instead of just p values. If you are dealing with symmetric confidence intervals, you can replace the upper and lower confidence bounds with a single number, the margin of error (i.e., half the width of the confidence interval, the "+ or -" term in the traditional formula for confidence intervals). So then you have one number to report, the MOE, for every possible pairing of groups in your anova, instead of two numbers (the upper and lower confidence bounds). The presentation can then be compressed even further by using a table in the form of a matrix. Label the rows and columns of the matrix to represent the groups in your anova. Above the diagonal, enter the point estimate of the difference on the dependent variable between the pair of groups in that row and column. Below the diagonal, enter the MOE for that difference. That yields one concise table for all the information it sounds like you are trying to present.
If your results include some significant differences and some non-significant ones, you could bold-face (or in some other way highlight) the significant entries. Or, you could run all the post-hoc tests, and use the traditional ns, *, **, and *** attached to the relevant between-group differences above the diagonal to indicate their approximate p-values. Then you would have packed even more information into a single table.
Well I found a partial solution. An effect size estimator like epsilon-squared, omega-square or Freedman's theta or Vargha and Delaney’s A could be used. However, I doubt if they accept this as replacement for p-values, but it could be an addition. Below an example with epsilon-squared.
boot
p values and effect size statistics convey different kinds of information, and so in no way should they be thought of as substitutes for one another.
What statistics are reported will to some extent be dictated by external forces, e.g. what a thesis advisor or journal expects. But in general, I advise routinely reporting both *p* values and an appropriate effect size statistic. Ideally include a confidence interval for the effect size statistic. Confidence intervals about some estimated parameter (e.g. the mean, the difference in means, and so on), might be a more-informative replacement for a p value.
No small set of statistics can replace all the information contained in the raw data. It's the task of the analyst to determine how to fairly convey the essence of the analysis with a few statistics and plots. There isn't necessarily a set formula for this.
Along with the p value and effect size statistic, I often advocate for the inclusion of some kind of assessment of "practical importance", which might include economic, social, or moral considerations, and may not follow obviously from the hypothesis test results or effect size statistic. That is, e.g. a small effect size may be very meaningful in cases of life and death or when large amounts of money are on the line.
That's just my take on things.
As a final note, note that the epsilon squared in that rcompanion function isn't really appropriate for anova. It's more reflective of a Kruskal-Wallis analysis. For an anova, usually eta-squared, partial eta-squared, or omega-squared is used. These are similar to r-squared. Depending on the purpose, tho, a plot showing the differences in group means --- with some measure of dispersion or confidence interval --- may be more informative for the reader.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA