LABOUR ARBITRATION DECISION

Ryerson University v Ryerson Faculty Association, [2018] 2018 CanLII 58446. https://canlii.ca/t/hsqkz

Decision limiting use Ryerson can make of SETs in T&P decisions.  See Freishtat and Stark Expert Reports, below.

ACADEMIC ARTICLES

Student evaluations are an intensely studied area, particularly in disciplines which make use of statistical tools.  The volume of published material is truly astounding. There are also a number of new studies which try to evaluate the impact of moving to online teaching on evaluations.  What follows is a very small selection.

Lavallee, Jaime, “How To Be Biased in the Classroom: Kwayeskastasowin - Setting Things Right?” (2022) 48:3 Mitchell Hamline Law Review, online: <https://open.mitchellhamline.edu/mhlr/vol48/iss3/3>.

A new-to-the-legal-academy female Cree-Metis teacher describes and analyzes her experience and approach teaching a new mandatory course in Indigenous law, and what she found in her course evaluations.  See also La Touche et al, below, on the question of impact on instructors.

Lazos, Sylvia R, “Are Student Teaching Evaluations Holding Back Women and Minorities?: The Perils of ‘Doing’ Gender and Race in the Classroom” in Gabriella Gutiérrez y Muhs et al, eds, Presumed Incompetent: The Intersections of Race and Class for Women in Academia (Utah State University Press, 2012).

An effort to summarize the plethora of studies suggesting problems with how students evaluate instructors who are women and/or minorities if they talk about gender and race in the classroom.

Abel, Richard L, “Evaluating Evaluations: How Should Law Schools Judge Teaching?” (1990) 40:4 Journal of Legal Education 407–465, online: <http://www.jstor.org/stable/42898120>.

An older piece from a CLS inclined professor.

Fisher, Warwick et al, “Student evaluations: Pedagogical tools, or weapons of choice?” (2020) 30:1 Legal Education Review, online: <https://ler.scholasticahq.com/article/14561-student-evaluations-pedagogical-tools-or-weapons-of-choice>.

Australian context.  

Ho, Daniel E & Timothy H Shapiro, “Evaluating Course Evaluations: An Empirical Analysis of a Quasi-Experiment at the Stanford Law School, 2000-2007” (2008) 58:3 Journal of Legal Education 388–412, online: <http://www.jstor.org/stable/42894079>.

An effort to evaluate reliability and validity on the occasion of a change in method of administration (to online) and change in wording of some questions. Concludes that these changes rendered the results under the old method and new method incompatible for comparison.

Kreitzer, Rebecca J. & Sweet-Cushman, Jennie (2022). Evaluating Student Evaluations of Teaching: a Review of Measurement and Equity Bias in SETs and Recommendations for Ethical Reform. Journal of Academic Ethics 20 (1):73-84. https://philpapers.org/rec/KREESE

Recent meta-study of evaluations,. Illustrates complex and multi-directional findings around gender and race in evaluation, indicating that downgrading female and/or POC instructors might not be the only impact that can be seen from SETs.

Lubicz-Nawrocka, Tanya & Kieran Bunting, “Student perceptions of teaching excellence: an analysis of student-led teaching award nomination data” (2019) 24:1 Teaching in Higher Education 63–80, online: <https://doi.org/10.1080/13562517.2018.1461620>.

A potentially useful illustration of how student responses to what “excellent” teaching is tend to cluster, using data from student-led teaching award nominations.  No gender/race analysis but perhaps helpful in understanding what students appreciate and why we might be helped by hearing positive feedback from students.  

Reverter, Antonio et al, “Unravelling student evaluations of courses and teachers” (2020) 7:1 Cogent Education 1771830, online: <https://doi.org/10.1080/2331186X.2020.1771830>.

A recent study looking at the evaluations at an Australian economics school concludes “The overall teaching rating awarded to academics clusters most with approachability and encouragement of student input—aspects of temperament and style—and not with explanatory skill or organisational ability.”

Tevis, Britt P & K E Powell, “Student Evaluations of Teaching: An Unlawful Barrier to Women’s Professional Advancement in Australian Universities” (2019) 37 39.

Australian article taking on the impact of teaching evaluations on professional advancement for women.

Cashin, William E. Student ratings of teaching: A summary of the research. Center for Faculty Evaluation & Development, Division of Continuing Education, Kansas State University, 1988.

An example of a study finding  that gender and race do not play a role in teaching evaluations. 

REPORTS

La Touche R, Kowalchuk L, Wijesingha R. (Re)Prioritizing Pedagogic Feedback: Faculty Experiences with Qualitative Comments from Student Evaluations of Teaching (SETs) A Report Prepared for the Canadian Sociological Association (CSA).; 2020. https://www.dropbox.com/s/jwojtf85rmuz5it/SET_Full_Report_REV-compressed.pdf?dl=0

Read with Lavallee, above. Focused on how faculty receive the qualitative comments made by students on SETs.

Freishtat, Richard L, Expert Report on Student Evaluations of Teaching (SET), by Richard L Freishtat (2016).

Expert Report prepared for the Ryerson arbitration

Stark PB. Expert Report on Student Evaluations of Teaching  (Faculty Course Surveys) Prepared for The Ryerson Faculty Association and The Ontario Confederation of University Faculty Association.; 2016.

Expert Report prepared for the Ryerson arbitration

Ontario Confederation of University Faculty Associations. OCUFA Briefing Note on Student Questionnaires.; n.d. https://ocufa.on.ca/assets/OCUFA-Briefing-Note-Student-Questionnaires.pdf

Report of the OCUFA Student Questionnaires on Courses and Teaching Working Group. Ontario Confederation of University Faculty Associations; 2019. https://ocufa.on.ca/assets/OCUFA-SQCT-Report.pdf

Ontario Faculty Association Agreements: Student Questionnaires and Peer Evaluation of Teaching. Ontario Confederation of University Faculty Associations; 2019. https://ocufa.on.ca/assets/SQCT-companion-FA-agreements.pdf

Survey of Ontario Faculty agreements and what they say about SETs / Peer Evaluation of teaching.

VIDEO

Validity and Equity Problems in Law School Teaching Evaluations, Faculty Conferences: Northwestern Pritzker School of Law.; 2022. Accessed June 12, 2022. https://www.law.northwestern.edu/research-faculty/events/conferences/teaching-evaluations/

Video of a recent panel discussion on this issue hosted by Northwestern.

REFORM

Dalhousie University, “Holistic Evaluation of Teaching Policy”, online: Dalhousie University <https://www.dal.ca/dept/university_secretariat/policies/academic/holistic-evaluation-of-teaching-policy.html>.

Description of an effort to revamp evaluations of teaching to reduce reliance on SETs.

WHY (how)  DO WE DO STUDENT EVALUATIONS?

How a selection of universities explain what they are doing when they do SETs.

Berkeley Law, “Teaching Evaluation Procedures”, online: Berkeley Law <https://www.law.berkeley.edu/academics/evaluation-procedures/>.

University of Ottawa, “Evaluation of teaching and courses”, online: University of Ottawa <https://www2.uottawa.ca/about-us/provost/evaluation-teaching-courses>.

University of Toronto Faculty of Law, “Course Evaluations | Academic Handbook”, online: <https://handbook.law.utoronto.ca/courses/course-evaluations>.

University of Western Ontario Faculty of Law, “Academic Policies and Procedures”, online: https://law.uwo.ca/current_students/student_services/academic_policies_and_procedures.html

York University, “Course Evaluations”, online: <https://courseevaluations.yorku.ca/facultyhelp/midcoursetraining/>.

NOT JUST FOR T&P: A reminder of how students use SETs sometimes…

“The Canons Big Book of Course Reviews: 2022-23 Edition”, Canons of Construction (University of Alberta Faculty of Law Student Newspaper) (17 March 2022), online: <https://www.canonsonline.com/2022/03/the-canons-big-book-of-course-reviews-2022-23-edition/>.