I am using SPSS and my model is: 3 raters, two-way mixed effects, absolute agreement. Data are far from normality, I do not know a non parametric procedure and I do prefer not to categorize the continuous data and so use kappa.
Hi, Amir.
In this study of reliability, three raters assess the abdominal aortic calcification in 100 plain radiographs with a 24-points scale. Range of distribution is from zero to 24.
Kolmogorov-Smirnov and F tests show that data are not normally distributed and the presence of heterocedasticity.
Hence, the assumptions of ICC are not fulfilled, and I wonder what is the best choice: log-transformation, categorization of the ratings and kappa, a non-parametric test...
Thank you both.
Bart: Interesting and useful paper, suggesting generalized linear mixed-effects models for non-Gaussian data.
Amir: These are the results of K-S test and the box-and-whisker graph. So, you also think of GLMM?
Kolmogorov-Smirnov
Statistic df Sig.
Rater 1 .154 100 .000
Rater 2 .149 100 .000
Rater 3 .148 100 .000
Shapiro-Wilk
Statistic df Sig.
Rater 1 .895 100 .000
Rater 2 .891 100 .000
Rater 3 .895 100 .000
Dear Amir:
I have chosen different upper limits and I have recoded the values according to them. The distributions were still quite skewed and p-value of K-S test was
Thank you, Jos, for the reference.
After your suggestions, I think of:
* Run log-transformation and ICC, for the continuous variable.
* Run Kendall´s coefficient of concordance or Cohen´s kappa for multiple raters, after ranking dataset.
In order to show both results.
Regards.
Hi, I have a similar question. I am calculating ICC as a measure of test retest reliability (using SPSS). I have transformed my data as it does not follow normal distribution. I have used square root transformation following which, data follows normal distribution. But, now while calculating SEM and MDC from ICC using formulas: SD of test√1-ICC for SEM and SEM X 1.96√2 for MDC. Do I need to take the data back into original form to calculate SEM and MDC? I am not sure about considering SEM and MDC value based on transformed data as it may not make sense? Any suggestions! Thanks
It seems to me that I can not use ICC calculated from a transformed data for SEM and MDC. Then, How far it is right to use cronbach's alpha to calculate SEM and MDC for a questionnaire data?
I am facing quite a similar problem with my data. My objective is to assess across-day test-retest of a procedure measured in a continuous scale. Data are far from normality. Transformation did not affect normality.
SPSS gives me the option to run a "Friedman's Chi-Square" instead of the standard parametric ANOVA. Is this the indicated procedure to deal with non-parametric data in this analysis? If no, how can I calculate ICC from non-parametric data?
- Badges
- Science topic
- Similar topics
- Medicine
- Physiology
- Biosignals
- Biomedical Signal Processing
- Statistical Analysis