I am interested in analyzing the quality of online instruction but would like to use a validated protocol to do so versus creating my own tool. Is there one you may know of for higher education?
What is the aspect of instruction you are interested in analyzing?
I have several publications related to using content analysis, as Claudia mentioned. I focus specifically on the Interaction Analysis Model and have an unpublished manuscript analyzing its validity. Another article, including one of the original authors of the tool, is also attached.
I have also attached a journal article (which just won an award, by the way) about faculty behaviors that promote critical thinking in peer interaction.
Let me know if these are useful, or if you are trying to take your research in a different direction.
Best wishes,
Barb
Article Assessing social construction of knowledge online: A critiqu...
Article In support of the Interaction Analysis Model for evaluating ...
Article An Analysis of Faculty Promotion of Critical Thinking and Pe...
At the end of every digital lesson some learning assessment activity has to be planned and the learning outcome is measured based on the score. Perhaps we have to design an environment coping with lot of online activities ranging from Multiple CHoice questions (MCQ), match the following, forum discussions, Fill in the blanks, digital journals etc.,
I think it is very important that you clarify what exactly are the variables or categories that you would like to study. Observations of online "classrooms" may focus on very different aspects: instructor's behavior, student-student interactions, cognitive engagement, quality of content, and many others. From my experience, the Community of Inquiry framework is a very useful model. You can take a look at the CoI questionnaire available at: https://coi.athabascau.ca/coi-model/coi-survey/
Try: Howell-Richardson C (2010) A pragmatic approach to analysing CMC discourse in R. Taiwo (ed.) Handbook of research on discourse behaviour and digital communication: Language structures and social interaction. Chapter 49: p759-775. Philadelphia. IGI Global.
You also might want to investigate Rourke, Anderson and Garrison's material on Communities of Practice, as well as their tools for assessing Social Presence (Rourke, Anderson, Garrison & Archer (1999). Assessing social presence in asynchronous text-based computer conferencing. Journal of Distance Education, 14(2), 50-71.
You can also check this review "Content analysis schemes to analyze transcripts of online asynchronous discussion groups: A review" by Bram De Wever et al. (2006). This paper rewiews different techniques used to analize online group discussions (content analysis). I think you'll find this useful. Best regards!
What is the aspect of instruction you are interested in analyzing?
I have several publications related to using content analysis, as Claudia mentioned. I focus specifically on the Interaction Analysis Model and have an unpublished manuscript analyzing its validity. Another article, including one of the original authors of the tool, is also attached.
I have also attached a journal article (which just won an award, by the way) about faculty behaviors that promote critical thinking in peer interaction.
Let me know if these are useful, or if you are trying to take your research in a different direction.
Best wishes,
Barb
Article Assessing social construction of knowledge online: A critiqu...
Article In support of the Interaction Analysis Model for evaluating ...
Article An Analysis of Faculty Promotion of Critical Thinking and Pe...
I think - as stated above - this depends on the particular area of instruction you're going to observe: e.g. classroom management - students' interactions - learning assessment - course design - course delivery - etc.
A simple query on GOOGLE SCHOLAR led to many interesting articles, books and research studies in this area..Wish you all the best:
At my university, everyone structures their courses based on the "Quality Matters!" rubrics. The course are also evaluated against this rubric during peer reviews.