Dear scholars,
As I use in my project data collected from individuals in work groups, I should according to theory test submitted hypotheses through a multilevel framework. Before doing so, I started examining interrater agreement and consistency statistics: rWG(J), ICC(1) and ICC(2). As a result, assessment of the rWG(J) (0.85) indicated strong agreement among group members on how they perceived internal conflicts. Further, the amount of variance in a respective variable that could be attributed to belonging to the higher-level unit (i.e., ICC(1) = 0.27) was high according to Woehr et al. 2015. Still, the reliability score of the construct representing the higher-level units (i.e., ICC(2) = 0.30, i.e., the group conflict) fell far below the acknowledged value of 0.70 (Klein and Kozlowski, 2000). Thus, my question is whether it would be acceptable (and advisable) to analyze these data at an individual (i.e. perceptual) level?
Hi Lukasz,
Unless there is a clear justification for analyzing your data from the individual level, the strict answer would be 'no' since you appear to be studying work groups (this would be an example of the ecological fallacy noted by Klein and Kozlowski, 2000, p. 213).
Thanks...Nick
Please refer to the difference between the sample size. The high-level sample is generally the level of expert method, so their consistency is high. The consistency of low level is relatively poor. The judgment depends on the user's needs. The key is that there is a comparability between the same sample size. Personal judgment, after all, has limitations.
In psychometric theories, individual level constructs can apply the random effect. In many cases of school-related research include original HLM articles use intelligence score.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics