Article Student Evaluations of Teaching Effectiveness: An Instrument...
I don't think that Serin (2019) adequately understands my criticisms of the use of SETs (Hornstein, 2017), which boil down to the inappropriate use and interpretation of descriptive statistics (one cannot legitimately calculate means from non-parametric data); one cannot apply statistical tests of significance on non-parametric data; sampling errors; and gender bias (negative bias against women in teaching evaluations). Arguably, SETs are nothing more than opinion surveys. If anything, they provide information about how satisfied students are with having taken the course, although even that is subject to interpretation.
Faculty members' careers have in many instances been negatively affected by these invalid instruments, which provide highly suspect evaluations.
The good: Student evaluations are one tool to make professors somewhat more accountable to the students that they teach.
The bad: It is largely a popularity contest. I have entered a large number of popularity contests and realized that they are generally not a good assessment. Often the winner and experts wonder why one piece of art wins and another does not. Yes, sometimes it is obvious, but as often not.
The ugly: It may result in poorer students if the faculty plays to the audience and does not give tough assignments that the students hate but end up being good for the student in the long run.
Hello Henry,
The technical concerns you raise are easily avoided:
1. Use medians instead of means (for ordinal data)
2. Yes, statistical significance tests exist for ordinal data, so that's not a restriction.
3. Sampling error? Most institutions attempt a census sample for SETs. Some students may well opt out, but that's true in any sample that involves people.
4. Gender bias (and other examples of what should be irrelevant influences) is a possibility, but it would not be restricted to SETs.
Studies of "effective teachers" have shown that some characteristics are, in fact, positively and consistently related to teaching efficacy. One simple example is enthusiasm for the subject matter (and in class). If a SET included a rating for this characteristic, many might judge that as a pertinent piece of information (as well as being an evidence-based indicator).
Now, the rhetorical questions:
Are SETs a measure of opinion/sentiment/judgment? Yes.
Does that make them useless? No.
Should people make decisions about a single round of SET ratings? No.
Should a consecutive set of poor SET ratings be a signal that some room for improvement/change exists? Yes.
Are students sometimes unfair in their ratings? Yes.
Are all students unfair in their ratings? No. (If anything, I have found that students are remarkably forgiving of mediocre instructional experiences in their ratings.)
Finally, the real question: What are defensible ways for SETs to be used? If the goal is instructional improvement, I would say this is one (of many) indicators that could be a part of that process.
I'm sure the discussion will continue; it usually does when people have strongly-held opinions!
Best regards.
You really think that the issues are easily avoided? Maybe, but there is little to show that the issues are being addressed appropriately. Few faculty appreciate the importance of the concerns. Ease of use still seems to trump everything else. Faculty do not demonstrate that decisions based on this ersatz data can have significantly negative consequences for tenure and promotion.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Applied Statistics