This is a question for all the statisticians out there! I am conducting an interrater reliability study. I have decided that using the Intraclass Correlation coefficient (ICC) would be the most appropriate statistical procedure, given that I have more than 2 raters involved (k=4). I have also calculated my appropriate sample size. My question relates to the type of model. I have chosen absolute agreement over consistency as I want to measure the ratings with absolute agreement to test the reliability of utilizing a proposed assessment rubric. I am not really sure if it is a two-way mixed or two-way random model, given that I have “randomly’ selected and invited raters from different disciplines, however, all four raters are fixed and will rate the same subjects drawn randomly from a larger population pool. The raters do not know the subjects. I am inclined to think this is a two-way mixed model rather than a two-way random model. What is the general consensus? I would greatly appreciate any feedback.
Hi,
For your question, Kindly see the following links:
https://www.youtube.com/watch?v=v2d6eyYoh4M
http://www.ats.ucla.edu/stat/spss/library/whichicc.htm
http://www.en.msc-epidemiologie.med.uni-muenchen.de/download/winter_term_2011_2012/epiresearchdesigns_articles/muellerbuttner02_12.pdf
Hi Senthilvel,
Thank you for those links. I found them most helpful.
Kind Regards,
Cherie Tsingos
Hi Cherie,
I am not sure which software package you are using, but I am attaching the Stata help file for -icc-. It does an excellent job of describing the various options you are asking about, and gives examples.
I hope you find this helpful.
Ariel
www.stata.com/manuals13/ricc.pdf
Hi Ariel,
I am using SPSS but I have used Stata in the past, so I am familiar with the software package. Thank you for this information. It was very helpful!
Cherie
Hi, the results of Kappa weighted is allmost near of ICC, so if you use Kappa weighted(in stata) it is not mistake.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size