In my research, I have used metaphorical elicitation tasks to investigate learners' conceptualization of speaking fluency. In the analysis phase, I used thematic analysis to make themes out of their provided responses to the elicitation tasks. In the next step, I want to conduct an inter-rater (inter-coder) reliability to increase my analysis's credibility. Any article to suggest for doing so?
"Any article to suggest for doing so?"
Mohammad Naghavian,
Since there has been much debate over the use and interchangeability of ICR and IRR in qualitative research, the following article might come in handy for your work. It expounds on both concepts and provides practical guidelines for carrying out an ICR assessment.
O’Connor, C., & Joffe, H. (2020). Intercoder reliability in qualitative research: Debates and practical guidelines. International Journal of Qualitative Methods, 19, 160940691989922. https://doi.org/10.1177/1609406919899220
It is an open-access article that can be downloaded through the publisher's Website on the link above or the RG link below.
Article Intercoder Reliability in Qualitative Research: Debates and ...
Good luck,
In their recent work, Braun and Clarke re-label the core version of their original approach as "Reflexive Thematic Analysis." From that perspective, it would not be appropriate to conduct inter-rater reliability. So, if you label your research as TA and then apply IRR, you are almost certain to get complaints from the reviewers.
I suggest you look at Braun and Clarke's current three-part segmentation of TA and determine which of the two code-based approaches is a better fit to what you are doing.
Mohammad Naghavian
I agree that the article recommended by Mohialdeen Alotumi is a strong resource for you.
If you would find it helpful to explore a tutorial on coding reliability measures, you might find this article to be useful:
Process guidelines for establishing intercoder reliability in qualitative studies
https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.857.2280&rep=rep1&type=pdf
Wishing you the best!
Brooke Hallowell
Mohialdeen Alotumi, Thank you very much Dr. Alotumi, and sorry for my belated response. I read the suggested article and learned many new things in relation to IRR and ICR. The practical guide was also clearly stated.
David L Morgan Thank you very much for informing me about reflexive TA, Professor Morgan. After seeing your message, I read three recent articles on TA. As you stated clearly, it seems that reflexive TA believes in subjectivity as an integral issue to the process of analysis. Indeed, as subjectivity is not seen as a risk for the analysis, the interrater reliability is found to be an unnecessary step to be taken.
Brooke Hallowell Thanks for providing such a useful comment on my question and sorry for my belated response. Your suggested article was highly clear regarding the steps to be taken and the important points to consider.
Mohammad Naghavian You state that "Indeed, as subjectivity is not seen as a risk for the analysis, the inter-rater reliability is found to be a necessary step to be taken." Is that you meant to say? I personally would conclude that IRR is not necessary (Or even desirable) for reflexive TA,
David L Morgan Ow I am sorry. I meant to write unnecessary but I ended up writing necessary by mistake.
Glad to hear that you, Mohammad Naghavian, found the article helpful!
Good luck,
Mohammad Naghavian I thought so.
You can use the "down arrow" at the bottom of your post to edit it.
If Inter-rater (Un)reliabilty is seen as an indicator of subjectivity/idiosyncrasy, you might find a way to make it useful.
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Probability
- Probability Theory
- Reliability