In Norway, the government proposes to reward good teachers. Any sensible reward (or implicitly also punishment) mechanism needs observations of good and/or bad teaching. Learning is a complex phenomenon. I find it hard to believe that teaching rewards leads to better student learning. What do you think?
Is it possible to measure teaching quality?
Dear Prof. Haugen,
Yes, it is possible to measure teaching quality, in term of its possibility. However, of course, the measurement may be a debatable dilemma among the teaching stakeholders following to the perspective differences of the parties, but the WHY points of the measurement may also need to be considered here. As it was elucidated in your question description, mechanism of the measurement and criteria of the quality may be two of the debate points.
Somehow, the measurement is needed in the context of its control function. For me, as a teacher, I think, I also need to be evaluated. In the different aspect, the teachers together with the students also learn in the learning process.
It is a personal opinion.
Regards,
Unfortunately, there will always be a lot of subjectivity in this assessment. How otherwise. After all, in fact, everyone considers themselves teachers (for children, employees, passers-by, friends ...).
One of the main struggles with measuring performance in higher education – whether of departments, faculties, or institutions – is how to measure the quality of teaching.
Teaching does not go entirely unmeasured in higher education. Individual courses are rated by students through course evaluation surveys, which occur at the end of each semester. The results of these evaluations do have some bearing on hiring, pay, and promotion (though how much bearing varies significantly from place to place), but these data are never aggregated to allow comparisons of quality of instruction across departments or institutions. That’s partly because faculty unions are wary about using individual professors’ performance data as an input for anything other than pay and promotion decisions, but it also suits the interests of the research-intensive universities who do not wish to see the creation of a metric that would put them at a disadvantage vis-a-vis their less-research-intensive brethren (which is also why course evaluations differ from one institution to the next).
Some people try to get around the comparability issue by asking students about teaching generally at their institution. In European rankings (and Canada’s old Globe and Mail rankings), many of which have a survey component, students are simply asked questions about the quality of courses they are in. This gets around the issue of using course evaluation data, but it doesn’t address a more fundamental problem, which is that a large proportion of academic staff essentially believes the whole process is inherently flawed because students are incapable of knowing quality teaching when they see it. There is a bit of truth here: it has been established, for instance, that teachers who grade more leniently tend to get better course satisfaction scores. But this is hardly a lethal argument. Just control for average class grade before reporting the score.
It’s not as though there isn’t a broad consensus on what makes for good teaching. Is the teacher clear about goals and expectations? Does she/he communicate ideas effectively? Is he or she available to students when needed? Are students challenged to learn new material and apply this knowledge effectively? Ask students those kinds of questions and you can get valid, comparable responses. The results are more complicated to report than a simple satisfaction score, sure – but it’s not impossible to do so. And because of that, it’s worth doing.
And even the simple questions like “was this a good course” might be more indicative than we think. The typical push-back is “but you can’t really judge effectiveness until years later”. Well, OK – let’s test a proposition. Why not just ask students about a course they took a few years ago, and compare it with the answers they gave in a course evaluation at the time? If they’re completely different, we can indeed start ignoring satisfaction types of questions. But we might find that a good result today is in fact a pretty good proxy for results in a few years, and therefore we would be perfectly justified in using it as a measure of teaching quality.
Students may be inexperienced, but they’re not dumb. We should keep that in mind when dismissing the results of teaching quality surveys.
http://higheredstrategy.com/how-to-measure-teaching-quality/
@ Haugen: may i suggest to send an E-Mail to my daughter
Professor for Grundschuldidaktik Sachunterricht - Naturwissenschaft & Technik
Universität Leipzig
Marschnerstraße 31
Haus III, Raum 034
Deutschland
She published research in that field and will be glad to help.
This is an area of personal study for me; and in fact am working on a paper for potential graduate studies which addresses this issue. The executive summary goes sort of like this: Learning is the "product" teachers try to deliver. To do this students are essentially raw materials; and not all of the same quality.
This means teachers must do the equivalent of mass customization to satisfy the customers: parents; students; and eventual employers. To me this is like a manufacturing process in that we have raw material, and processes which change the raw material into more learned material.
It is not rocket science to see right away that the number of variables may be overwhelming. In fact it seems amazing that learning takes place at all except that the raw material has a brain. I have not compiled much data on the processes and will before I move forward with the subject.
For now, I would say the answer to this question is no: instead we should try to measure learning since teachers have no control over the raw material quality and not much control over the learning process.
Lastly, I believe measures taken to "rate" teachers can only be subjective. To believe otherwise seems to me to be emotional folly.
A person can measure and evaluate everything. Even the world around the person. But who will be an objective judge of the deed. Therefore, when assessing the level of teaching quality, it is advisable to take into account, first of all, professionalism, the ability to explain complex concepts, an individual approach to students and much more.
Whether it is possible to measure (in a true sense) teaching quality is highly debatable. In this discussion on RG, I appreciate the input from Gregory.
Indeed, it is about how education, teaching and learning are being positioned. It is an ontological question. Dominant metaphors for education, teaching and learning in policy documents and public discourse are "production", "service", which ensue "customer satisfaction", etc. . These metaphors make us perceive the need for assessment and measurement despite some obvious follies in such endeavours.
But are these "production"/"service" metaphors germane? A more adequate metaphor for education, teaching and learning should not take the essential element from this human endeavour -- its dialogical nature. The "dialogue" metaphor -- in an encompassing Bakhtinian sense --- encourages a clearer thinking about the ontology of education and could perhaps translate into more thoughtful educational policies and better teaching practice.
Indeed, how do we assess the quality of this "dialogue"?
Thanks to everybody for engaging in an important discussion. In economic theory we use to say: The harder something is to measure, the more careful one should be to introduce reward systems. In my opinion nothing is harder to measure than teaching quality.
How to Measure Teaching Quality
by Alex Usher
One of the main struggles with measuring performance in higher education – whether of departments, faculties, or institutions – is how to measure the quality of teaching.
Teaching does not go entirely unmeasured in higher education. Individual courses are rated by students through course evaluation surveys, which occur at the end of each semester. The results of these evaluations do have some bearing on hiring, pay, and promotion (though how much bearing varies significantly from place to place), but these data are never aggregated to allow comparisons of quality of instruction across departments or institutions. That’s partly because faculty unions are wary about using individual professors’ performance data as an input for anything other than pay and promotion decisions, but it also suits the interests of the research-intensive universities who do not wish to see the creation of a metric that would put them at a disadvantage vis-a-vis their less-research-intensive brethren (which is also why course evaluations differ from one institution to the next).
Some people try to get around the comparability issue by asking students about teaching generally at their institution. In European rankings (and Canada’s old Globe and Mail rankings), many of which have a survey component, students are simply asked questions about the quality of courses they are in. This gets around the issue of using course evaluation data, but it doesn’t address a more fundamental problem, which is that a large proportion of academic staff essentially believes the whole process is inherently flawed because students are incapable of knowing quality teaching when they see it. There is a bit of truth here: it has been established, for instance, that teachers who grade more leniently tend to get better course satisfaction scores. But this is hardly a lethal argument. Just control for average class grade before reporting the score.
It’s not as though there isn’t a broad consensus on what makes for good teaching. Is the teacher clear about goals and expectations? Does she/he communicate ideas effectively? Is he or she available to students when needed? Are students challenged to learn new material and apply this knowledge effectively? Ask students those kinds of questions and you can get valid, comparable responses. The results are more complicated to report than a simple satisfaction score, sure – but it’s not impossible to do so. And because of that, it’s worth doing.
And even the simple questions like “was this a good course” might be more indicative than we think. The typical push-back is “but you can’t really judge effectiveness until years later”. Well, OK – let’s test a proposition. Why not just ask students about a course they took a few years ago, and compare it with the answers they gave in a course evaluation at the time? If they’re completely different, we can indeed start ignoring satisfaction types of questions. But we might find that a good result today is in fact a pretty good proxy for results in a few years, and therefore we would be perfectly justified in using it as a measure of teaching quality.
Students may be inexperienced, but they’re not dumb. We should keep that in mind when dismissing the results of teaching quality surveys.
http://higheredstrategy.com/how-to-measure-teaching-quality/
Hello Kjetil - this is something that the UK is now implementing through the TEF - see the link below. 299 universities, colleges and private institutions have 'signed up'.
However, we are already acutely aware of the consequences of measuring education - the same body that is setting up the TEF reviewed the REF (research excellence framework).
It is indeed a very political space.
Kind regards, Tony.
http://www.hefce.ac.uk/lt/tef/
http://www.hefce.ac.uk/pubs/rereports/year/2015/metrictide/
The answer is no; until you decide on the objective function you are trying to optimize. Students can like or dislike a class or a prof for a host of reasons that have little to do with "teaching." If they love the class it doesn't necessarily mean you taught well. If they hate the class is doesn't necessarily mean you taught poorly.
Check out Rate Your Professor or something similar and you'll see the problem.
Spot on, Kjetil; really difficult to measure if you don't even know what you're measuring.
Very good points Peter V. Bias, in close correspondence with my own view on the matter. all this counting and measuring - where will it end, the death of academia?
. The answer is no. As the students can you or doesn't like the dynamics of the class mean you taught wrong either. What more important is that you get to the target.
This is a fun topic.
I remember many years ago our department wanted to know what the relationship was between our students' grades on a standardized test normed by students all over the country and the grades received in our classes (and the students' SAT scores as a control). I did the 'study' to be presented to the department faculty.
There was no correlation between students' grades and their standardized test scores. However, there was a positive correlation with SAT scores .
Moreover, there was a very positive relation between students' grades in a course and the 'satisfaction' with the professor for that course (measured by the average of about 12 different components). Better grades, better prof (though, of course, not a perfect relation).
Unfortunately, students (and some profs) are "human."
Its more like in the case of teaching rewards lead to better student learning as a result of the teacher being motivated. Perhaps, teachers need constant motivation besides rewards per se in order to evade from being stagnated in their professions.
I think that the basic idea of the teaching is receipt knowledge to the students to be able to benefit from it in the future.
Therefore, if I want to know who is a good teacher, I should watch the students in their works after graduate.
The teacher is a factory. If you want to assess a factory, look to its products quality.
In general I have a strong feeling that politicians and academia administrators have a tendency to overemphasise the role of the lecturer in the learning process. Of course very understandable, after all, if the lecturer is not that important, why should we get public money for teacher growth. Personally, I have a strong feeling that the teacher/lecturer's role is much more of a facilitator. Learning takes place between students much more than in the teacher/student relation. In such a perspective; a incentive system for excellent teaching seems to me completely useless. Not to speak about the obvious quality measurement problems.
the answer is rather not clear because, teaching quality cant be judged just on feedback from.student and senior teachers. Its not necessary that to be a good teacher only students should be assessed. Rather, a lot of factors contribute for quality teacher.or a good teacher
Performance measurment for teachers is difficult, so consider the evalution measures for teachers, and what is important is the extent to which students accept the knowledge of the teacher in addition to the diversity of teaching methods.
- Badges
- Science topic