I have a question about how to calculate the exact percentage agreement as well as the adjacent agreement (agreement when scale values are either the same or differ by ±1) manually.

Scenario:

100 raters have assessed a single item on a 5-point scale. Common methods for determining interrater reliability do not work because I only have one item. Therefore, I want to calculate the percentage agreement manually.

Fictional Results:

• Response 1 = 20 raters
• Response 2 = 20 raters
• Response 3 = 50 raters
• Response 4 = 10 raters
• Response 5 = 0 raters

Could someone help me with the formula to calculate this, especially when there are so many raters involved? Thank you!