Every few years someone does a study that proves student evaluations in higher education are biased against people of color, white women, and women of color. The latest study also shows that untenured faculty are likely to receive lower scores for the same work. And, also no surprise, that student perception of inflated grades (ie that they are earning an A no matter what work they do or do not do) will increase the evaluation.
The interesting thing about these studies, is it does not matter how “objective” the agency administering them or financing them is, schools continue to rely on evaluations unweighted by considerations of diversity. Evaluations are also consistently used to prove “lack of fit” during the tenure process. Though some of us, myself included, have managed to find ways to make even the most contentious classroom fill out the forms in a positive light the biases involved are no surprise to any of us. So the question then becomes: why are they still in use?
This from the Chronicle including a link to pay for a copy of the most recent study:
Student evaluations of their instructors can often reflect certain biases that professors themselves have no control over, says a study by Michael A. McPherson and R. Todd Jewell, both associate professors of economics at the University of North Texas. Given the weight that such evaluations receive in decisions regarding tenure and promotion, they ask, should evaluation scores be adjusted?
According to the study — which is based on student evaluations from 280 master’s-level courses in economics at North Texas from 1994 to 2005 — faculty members are more likely to receive lower scores from students if their course meets just once a week, or if they are teaching a theory course. Faculty members who are nonwhite also earn lower average scores, says the study, as do professors who are nontenured.
In addition, say the authors, instructors are able to increase their average evaluation scores by inflating the grade expectations of their students.
Adjusting evaluations in a way that controls for factors beyond the control of faculty members — such as their race — could make the ranking process fairer, as well as more accurate, the authors argue. After adjusting for such factors in their own study, the authors report that there are “clear differences between raw and adjusted rankings and that these differences are substantial for some instructors.” In one of their adjustments, a nontenured female instructor went from being ranked 10th out of 17 tenure-track instructors, to being the top-rated instructor.
“The issue of bias in student-evaluation-of-teaching scores is one that each department should discuss, and one that each department may resolve in its own fashion,” conclude the authors. They add that, in general, “departments may find that adjustments based on some set of criteria are a valuable exercise.”
The article — “Leveling the Playing Field: Should Student Evaluation Scores Be Adjusted?” — is available to subscribers or for purchase through Blackwell Synergy.