I have read that Fleiss´ kappa measures multi-rater agreement among any constant number of raters (≥3) when different subjects are rated by different subset of raters. However, in our study the same 3 raters classify all the subjects. There are published studies with our same design and they use Fleiss kappa, but I´m not sure that it´s correct or is better to use Light´s kappa (compute Cohen´s kappa for each coder pair then averaged manually). I´ll appreciate any advice, thank you.
Thank you very much for your suggestions! I think I am going to finally use Fleiss´ Kappa.
Kind regards
I know this might be a bit late but you might also consider using mKAPPAfree which can be used for multiple raters assuming you have free marginals. More info (and a java calculator) can be found here: http://justusrandolph.net/kappa/
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics