I would like to know the way to calculate the sample size of a study to determine the agreement (weighted Cohen's Kappa index) between two thecniques.
Hello Anna,
Assuming you're willing to think in SD units as the degree of difference between two kappa estimates, and that the two estimates are from independent samples, then the usual formulas for independent t-test would be pretty good indicators of requisite N (or n per sample). Thus, you could use G*Power (free for download & use: http://www.gpower.hhu.de/en.html) to estimate.
Note that SD values for kappa depend on the magnitude of the two values, po--proportion agreement observed and pe--proportion agreement expected, and are maximized as (po -> 0.5 and pe -> 1.0), and minimized when (po -> 0 or 1 and pe -> 0). See https://ncss-wpengine.netdna-ssl.com/wp-content/themes/ncss/pdf/Procedures/PASS/Confidence_Intervals_for_Kappa.pdf) for older (Cohen's) and newer (Fleiss et al.) estimation formulae for the SD of kappa.
Finally, the R library, kappaSize, was tailored to assist in sample size determinations for tests involving kappa (and is free).
Good luck with your work!
Dear Dr. Arnau,
Prof. Morse very nicely elaborates this issue. Apart from that, I want to add two attachemets that may be helpful to you.
https://www.ime.usp.br/~abe/lista/pdfGSoh9GPIQN.pdf
ebph.it/article/download/12267/11379
Thanks.
Dear Anna Arnau ,
I found this that might help you.
Give a look to the manuscript and see this link too (it's the calculator):
https://wnarifin.github.io/ssc/sskappa.html
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size