If we are doing a Grounded Theory research involving coding of interview transcripts as propounded by Glaser (1978) and Strauss and Corbin (1990), then:
1. Is there a heuristic to decide on the coders for a qualitative research?
2. How is the inter-coder reliability calculated in such a case?
GT as a methodology emphasizes reflexivity, theoretical sensitivity, and "circular" testing of your codes, concepts, and categories as key strategies to ensure rigidity and quality. In my view, the idea of intercoder reliability does not make much sense against this epistemological background.
The whole idea of coding is very different in GT than in content analysis or quantitative text analysis. In other words, "coding" can mean very different things. Coding in GT signifies the whole process of theory building, not so much (or only) the process of assigning codes to pieces of text. Hence, the calculation of a reliability index is technically complicated (impossible) because you can't simply compare code-text-assignments between different persons working with the data.
I'm not sure HOW it's calculated, but I'm using Dedoose software to do the coding, and it can calculate inter-coder reliability for you. It allows different coders to code "blind" to the others and then you can do the calculations. I believe there are explanations about how they calculate it on their website (just google "Dedoose"). That might be enough info to do it on your own (you'll have to check), but if you want to use Dedoose, it's about $11 US per user per month. Personally, I think it's totally worth it. This is my first time doing qualitative research, and I find it to be pretty user friendly.
GT as a methodology emphasizes reflexivity, theoretical sensitivity, and "circular" testing of your codes, concepts, and categories as key strategies to ensure rigidity and quality. In my view, the idea of intercoder reliability does not make much sense against this epistemological background.
The whole idea of coding is very different in GT than in content analysis or quantitative text analysis. In other words, "coding" can mean very different things. Coding in GT signifies the whole process of theory building, not so much (or only) the process of assigning codes to pieces of text. Hence, the calculation of a reliability index is technically complicated (impossible) because you can't simply compare code-text-assignments between different persons working with the data.
I agree with Kenneth Horvath that inter-rater reliability is not appropriate in GT, and that it is a better fit to qualitative content analysis.
As both he and Kathryn Plaisance make clear, any calculation of inter-rater reliability requires two coders to apply the same codebook to a data set. If that is indeed what you want to do, then there are a number of different coefficients that you can calculate, such as Cohen's kappa and Krippendroff's alpha.
Thanks all. I agree with Kenneth and David and their explanation is what I was seeking. Thanks again for the clarification.
- Badges
- Science method
- Similar topics
- Methodology
- Research Methodology
- Qualitative Research