I am working with a team who are undertaking observations of nurses activities and recording timings using a coded activity, we have developed this ourselves using JAVA. We can export sorted activity by timings and produce comparison between observers.
Usually we use Cohen's Kappa to compare two observers as it allows to measure how much two people agree: http://en.wikipedia.org/wiki/Cohen's_kappa
Reliability and validity are not exactly the same thing, though.
Dear Graham Wright, Check the following WEB links for their usefulness.
http://www.academia.edu/458025/The_place_of_inter-rater_reliability_in_qualitative_research_an_empirical_study
http://www.ncbi.nlm.nih.gov/books/NBK92287/
If you go for Cohen's Kappa, be aware of its problems, such as the fact that it takes systematic disagreements to be agreements, and is rather sensitive to skewed distributions. An alternative could be Krippendorf's alpha. But what is it that you want to prove? Because from your description, maybe a correlation test (student, wilcoxon etc) would suffice...
- Badges
- Science topic
- Similar topics
- Filing