I have a dataset that includes 1900 companies. Also, I investigated 10 employees for each company. There is a question about the risk preference of each employee. At now, I need to calculate the ICC1 and ICC2 values for each company. I have already coded for each company, so each company will have a unique company_id. At now, I have the employee dataset, it means I have the 19000 data, and each employee will match the company according to the company_id. In this case, how to get the ICC1, and ICC2 value of each company in R. I have already tried for few days, expecting someone could resolve my problem.
Your question does not make sense to me. The ICC (regardless if 1 or 2, or ICC(1,1) vs. ICC(1,k) in other terminology) is a scalar computed from your data. It relates the between group variance to the within group variance or total variance. Therefore, you get a single value and NOT an ICC for each group/company. Please clarify your request.
Rainer Duesing Many thanks for your response. I understand that typically, ICC (Intraclass Correlation Coefficient) is calculated as a single scalar value for a dataset, reflecting the proportion of variance between groups compared to the total variance. However, my research scenario is a bit unique.
In my study, I am dealing with a dataset that includes 1,900 companies, with 10 employees surveyed in each company. Each employee has responded to a question regarding their risk preference. My objective is to assess the variability of these risk preference responses both within and between these companies.
For this purpose, I am interested in calculating ICC values for each company individually rather than a single ICC value for the entire dataset. Essentially, I aim to understand how the risk preference varies within each company (as a group) and then compare these variations across different companies. This approach requires computing an ICC value (both ICC1 and ICC2) for each company, using their unique company_id to segregate the data.
I am looking for a method to apply ICC calculations to each company separately, based on their unique identifiers in the dataset. Any advice or guidance on how to approach this in R would be greatly appreciated.
Pragmatically, if your are interested in the within group variance, why dont you calculate it simply for each company?? But as far as I know, you cannot calculate individual ICCs, since you only have one between group variance and this is part of the nominator and denominator. And especially in ICC2, the within group variance is multiplied by k-1 groups, so that you get the "group-mean reliability", which is per definition not a single group. Hence, I doubt that your idea goes in the right direction.
On the other hand, maybe you are looking at the wrong indices. Where ICCs charachterize the interrater "reliability", there is also the class of indices for "agreement" which is often used synonymously, but they are not (sometimes you will find the terms "consisteny" vs. "consensus" or something similar).
Have a look at indices like rwg and rwg(j) (for an overview and comparison of ICC and rwg , see Castro (2002)), as well as alternatives r*wg(J) by Lindell et al. (1999). They can be calcualted for each group and may be the index you are looking for.
Castro, S. L. (2002). Data analytic methods for the analysis of multilevel questions: A comparison of intraclass correlation coefficients, rwg (j), hierarchical linear modeling, within-and between-analysis, and random group resampling. The Leadership Quarterly, 13(1), 69-93.
Lindell, M. K., Brandt, C. J., & Whitney, D. J. (1999). A revised index of interrater agreement for multi-item ratings of a single target. Applied Psychological Measurement, 23(2), 127-135.
P.S.: Paul Bliese has a multilevel tutorial for R, where he shows how to calcualte the above mentioned indices, as well as others, since all have their specific problems, which would lead too far to discuss them here.
https://cran.r-project.org/doc/contrib/Bliese_Multilevel.pdf
- Badges
- Science topic
- Similar topics
- Methodology
- Research Methodology
- Research Papers