ICC to measure internal consistency of a scale is estimated using two-way ANOVA as I understand. But ANOVA has assumptions of normality of data & uncorrelated errors. Still, I read that ICC can be used for interval categorical data such as Likert scale. But what if my data are skewed (responses have ceiling and floor effect)? Also, most importantly, what do I do with clustering in this case?
Hi, Elina:
To analyze the internal consistency in a Likert scale can use the Cronbach's alpha coefficient. Is equivalent to the ICC when analyzing one sample.
Thanks, Jose! But I thought Cronbach's alpha was for continuous scales. Also, I have a bit of piling in my data set, so maybe Cronbach's is OK if one has more or less normal distribution, but I have 70% of response in categories 3-4 on a four-point Likert scale. Do you think it's still OK for me to use Cronbach's? I see ICC is also not appropriate since it's based on ANOVA which has strong assumptions. What would you advise for data such as mine? Thank you!
If your scale has ceiling and floor effects, then it is more important to look into that than whether to use ICC. Without knowing more about the scale or its purpose, is is difficult to say any more.
ICC is used for interrater reliability, that is when 2 or more raters are assessing some variable(s). ICC for consistency, average measures is the same as Cronbach's alpha. ICC for absolute agreement and consistency is used in preference to Cronbach's alpha (Cronbach, L. J., & Shavelson, R. J. (2004). My current thoughts on coefficient alpha and successor procedures. Educational and Psychological Measurement, 64(3), 391–418. doi:10.1177/0013164404266386).
A method that could discover why the scale has ceiling/floor effects is generalizability theory (Brennan, R. L. (2001). Generalizability theory. New York: Springer).
- Badges
- Science topic