Helloš
I have a question regarding Intraclass Correlation Coefficients (ICCs). My research is investigating the test-retest reliability of a neuropsychological test over five different test administrations one year apart. The test consists of five composites. In addition to using a repeated-measures ANOVA to investigate the significance of score change over the five years on all five composites, I wanted to use ICCs as a statistic for test-retest reliability as other test-retest reliability studies in this area use them. However, all other studies are over three test administrations and produce ICCs for times 1-2, 2-3, and 1-3 on all five composites. I have five test sessions and wanted to produce ICCs for 1-2, 1-3, 1-4, 1-5, 2-3, 3-4, 4-5 on all five composites. Thatās 7 different comparisons/correlations (Iām not sure which it is). Would this be possible? I read that the ICC model is based on an ANOVA model, so I am concerned about Bonferroniās correction. Does the error Bonferroniās corrects applies when using ICCs?
Any guidance would be greatly appreciated.
Not unless your primary focus is rank-order correlation/agreement (I.e., strictly ordinal-level
measurement data).
Hope this helps,
Matt
Well, for reproducibility purposes I use Lin's concordance coefficient (it is related to intracorrelation correlation). I attach a reprint to show its use
Hello Marcelle,
I guess it depends on the purpose of your ICC's. If you only want to use them descriptively as a measure for your retest-reliability, multiple ICC calculation should be fine. Usually people aim for values higher than 0.7 - 0.9 (depending on the discipline). Otherwise you would probably change or revise your test.
If, however you want to take it one step further and look at the p-values of your ICC's, I would indeed suggest to correct for the multiple comparison, because as you use your data multiple times, they are not independent anymore.
Best,
Steffen
ICCs are like Pearson Correlations (when two variable are considered). Generally, using ICC for ordinal measurements are not correct. However, for ordinal measurement with more than 10-15 levels, ICCs are used.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA