If the upper 95% CI of Cohen's kappa is calculated on SPSS as >1, how should it be reported, since the max meaningful value is 1? SPSS does not provide the output, but it can be derived from the SE. Since Cohen's kappa can not be >1, if the calculation leads to an upper CI of >1, how should this be reported? Should it be truncated at 1, or the value, e.g. 1.05, be reported as the upper CI, even if it is meaningless?
The calculation of confidence intervals for kappa is one of those problems that has bothered people for years. Have a look at this paper :
Article Confidence Intervals for the Kappa Statistic
The authors state "Although quite computationally intensive and time consuming, the bootstrap method may be an interesting option to calculate confidence intervals when variables have more than two levels; if three or more raters/replications are involved; and, above all, when a weighted kappa is envisaged."
You can use R-package ‘rel’, where a more precise method is implemented.
Hello Pieter,
If you're content with the CI estimation method, then you'd truncate the upper limit of the reported value at 1.0, and report it as such.
As Ronán Michael Conroy correctly indicates, common formulae for kappa CI estimates aren't infallible, especially at the extremes.
Good luck with your work.
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software
- SPSS