On June 28, William Kaplan, the Arbitrator appointed to resolve the issue of Faculty Course Surveys (FCS) and related matters, including the use of student evaluations of teaching (SET) for promotion and/or tenure decisions, between the Ryerson Faculty Association and the Ryerson University, rendered a decision which highlights the limitations of the use of SETs, and the importance of a well-rounded review of faculty teaching.
Evidence presented during the arbitration demonstrated that individual characteristics including race, gender, accent, age and the appearance of a professor, all influenced the results of SETs. Other factors such as whether the evaluations are done online versus in-class, the courses are elective or mandatory, the number of students in the course, the subject matter, and the teaching style also impact the SET results.
“The expert evidence led at the hearing persuasively demonstrates that the most meaningful aspects of teaching performance and effectiveness cannot be assessed by Student Evaluations of Teaching. Insofar as assessing teaching effectiveness is concerned – especially in the context of tenure and promotion – Student Evaluations of Teaching are imperfect at best, and downright biased and unreliable at worst.”
– William Kaplan
At the University of Ottawa, the use of A-Reports (based on student questionnaires) is one of several elements considered in the promotion and/or tenure process. While the overall procedure to evaluate teaching at Ryerson University and at our own institution differ in many ways, the Kaplan arbitration ruling presents valid criticism, and exposes limitations of some aspects of our system.
What role do the A-reports play in the evaluation process at uOttawa? Unlike many other institutions, A-Reports may only be used as an indicator towards a decision by the Dean to initiate or not a Direct Peer Review of Teaching (DPRT). As per article 24.2.1.3 c) of our Collective Agreement, unless the Dean initiates a DPRT due to a pattern of weak A-Reports or outstanding teaching, any evaluation of teaching must be considered to have met expectations. Thus, at our institution, the Central Administration cannot deny tenure or promotion based on “weak” A-Reports alone. While we appreciate that in our Collective Agreement, there are measures in place to allow professors to communicate their own reflections on teaching and learning in light of A-reports, we also recognize that this doesn’t fully address the inequities and consequences for professors who face strong (conscious or unconscious) bias in their teaching evaluations. In light of the low participation rates on online student evaluation of teaching, the APUO is further concerned about the possibility of low participation rates exacerbating bias and compromising the relevance of the data. This is particularly important for professors who plan to apply for promotion and /or tenure in the coming years.
As a result of the most recent round of collective bargaining, the APUO will meet with the Central Administration in May of each academic year to discuss and review issues arising from the use of student evaluations data in making career recommendations and decisions.
The Central administration mandated the Senate Committee on teaching and its evaluation to study approaches to evaluating teaching. The Committee will make recommendations to revise/change current practices. Even if the Senate accepts to modify the current practices, the Central Administration will be required to get the APUO’s approval prior to implementing its use to officially evaluate a member’s teaching.
We wish to remind members that they are strongly encouraged to raise concerns they have regarding the use of student evaluations of teaching with the APUO, mainly if they believe they might negatively impact their application for promotion and/or tenure.
You may also be interested in reading the Canadian Association of University Teachers’ (CAUT) November bulletin The end of student questionnaires?