Hello Reza,

One approach would be to use a linking design, in which multiple raters assessed some common core group ("anchor group") of participants. Discrepancies in the ratings of these could be used to adjust estimates for the remaining ratings.

There is a pretty good literature on linking designs, as well as multiple matrix sampling designs, both of which are viable mechanisms for dealing with incomplete data sets, if planned for a priori.

Good luck with your work.

Hi Reza,

have you seen this online course by M. Wu?

https://www.edmeasurementsurveys.com/

I found that very helpful, and it also contains a section on linking mechanisms.

All the best!

thank you for your answer dear David.

can you recommend any studies, or papers, at the top off your head?

David Morse

thank you dear Armin. I will check it.

Armin Jentsch