a short reading list on Student Evaluation of Teaching
Student Evaluations of Teaching: A Short, Annotated, List of Resources
June 2022
Download a .docx version here.
(important additions? write to [email protected] or [email protected])
LABOUR ARBITRATION DECISION
|
|
Ryerson University v Ryerson Faculty Association, [2018] 2018 CanLII 58446. https://canlii.ca/t/hsqkz
|
Decision limiting use Ryerson can make of SETs in T&P decisions. See Freishtat and Stark Expert Reports, below. |
ACADEMIC ARTICLES Student evaluations are an intensely studied area, particularly in disciplines which make use of statistical tools. The volume of published material is truly astounding. There are also a number of new studies which try to evaluate the impact of moving to online teaching on evaluations. What follows is a very small selection.
|
|
Lavallee, Jaime, “How To Be Biased in the Classroom: Kwayeskastasowin - Setting Things Right?” (2022) 48:3 Mitchell Hamline Law Review, online: <https://open.mitchellhamline.edu/mhlr/vol48/iss3/3>. |
A new-to-the-legal-academy female Cree-Metis teacher describes and analyzes her experience and approach teaching a new mandatory course in Indigenous law, and what she found in her course evaluations. See also La Touche et al, below, on the question of impact on instructors. |
Lazos, Sylvia R, “Are Student Teaching Evaluations Holding Back Women and Minorities?: The Perils of ‘Doing’ Gender and Race in the Classroom” in Gabriella Gutiérrez y Muhs et al, eds, Presumed Incompetent: The Intersections of Race and Class for Women in Academia (Utah State University Press, 2012). |
An effort to summarize the plethora of studies suggesting problems with how students evaluate instructors who are women and/or minorities if they talk about gender and race in the classroom. |
Abel, Richard L, “Evaluating Evaluations: How Should Law Schools Judge Teaching?” (1990) 40:4 Journal of Legal Education 407–465, online: <http://www.jstor.org/stable/42898120>. |
An older piece from a CLS inclined professor. |
Fisher, Warwick et al, “Student evaluations: Pedagogical tools, or weapons of choice?” (2020) 30:1 Legal Education Review, online: <https://ler.scholasticahq.com/article/14561-student-evaluations-pedagogical-tools-or-weapons-of-choice>. |
Australian context. |
Ho, Daniel E & Timothy H Shapiro, “Evaluating Course Evaluations: An Empirical Analysis of a Quasi-Experiment at the Stanford Law School, 2000-2007” (2008) 58:3 Journal of Legal Education 388–412, online: <http://www.jstor.org/stable/42894079>. |
An effort to evaluate reliability and validity on the occasion of a change in method of administration (to online) and change in wording of some questions. Concludes that these changes rendered the results under the old method and new method incompatible for comparison. |
Kreitzer, Rebecca J. & Sweet-Cushman, Jennie (2022). Evaluating Student Evaluations of Teaching: a Review of Measurement and Equity Bias in SETs and Recommendations for Ethical Reform. Journal of Academic Ethics 20 (1):73-84. https://philpapers.org/rec/KREESE |
Recent meta-study of evaluations,. Illustrates complex and multi-directional findings around gender and race in evaluation, indicating that downgrading female and/or POC instructors might not be the only impact that can be seen from SETs. |
Lubicz-Nawrocka, Tanya & Kieran Bunting, “Student perceptions of teaching excellence: an analysis of student-led teaching award nomination data” (2019) 24:1 Teaching in Higher Education 63–80, online: <https://doi.org/10.1080/13562517.2018.1461620>. |
A potentially useful illustration of how student responses to what “excellent” teaching is tend to cluster, using data from student-led teaching award nominations. No gender/race analysis but perhaps helpful in understanding what students appreciate and why we might be helped by hearing positive feedback from students. |
Reverter, Antonio et al, “Unravelling student evaluations of courses and teachers” (2020) 7:1 Cogent Education 1771830, online: <https://doi.org/10.1080/2331186X.2020.1771830>. |
A recent study looking at the evaluations at an Australian economics school concludes “The overall teaching rating awarded to academics clusters most with approachability and encouragement of student input—aspects of temperament and style—and not with explanatory skill or organisational ability.” |
Tevis, Britt P & K E Powell, “Student Evaluations of Teaching: An Unlawful Barrier to Women’s Professional Advancement in Australian Universities” (2019) 37 39. |
Australian article taking on the impact of teaching evaluations on professional advancement for women. |
Cashin, William E. Student ratings of teaching: A summary of the research. Center for Faculty Evaluation & Development, Division of Continuing Education, Kansas State University, 1988.
|
An example of a study finding that gender and race do not play a role in teaching evaluations. |
REPORTS
|
|
La Touche R, Kowalchuk L, Wijesingha R. (Re)Prioritizing Pedagogic Feedback: Faculty Experiences with Qualitative Comments from Student Evaluations of Teaching (SETs) A Report Prepared for the Canadian Sociological Association (CSA).; 2020. https://www.dropbox.com/s/jwojtf85rmuz5it/SET_Full_Report_REV-compressed.pdf?dl=0 |
Read with Lavallee, above. Focused on how faculty receive the qualitative comments made by students on SETs. |
Freishtat, Richard L, Expert Report on Student Evaluations of Teaching (SET), by Richard L Freishtat (2016).
|
Expert Report prepared for the Ryerson arbitration |
Stark PB. Expert Report on Student Evaluations of Teaching (Faculty Course Surveys) Prepared for The Ryerson Faculty Association and The Ontario Confederation of University Faculty Association.; 2016. |
Expert Report prepared for the Ryerson arbitration |
Ontario Confederation of University Faculty Associations. OCUFA Briefing Note on Student Questionnaires.; n.d. https://ocufa.on.ca/assets/OCUFA-Briefing-Note-Student-Questionnaires.pdf |
|
Report of the OCUFA Student Questionnaires on Courses and Teaching Working Group. Ontario Confederation of University Faculty Associations; 2019. https://ocufa.on.ca/assets/OCUFA-SQCT-Report.pdf |
|
Ontario Faculty Association Agreements: Student Questionnaires and Peer Evaluation of Teaching. Ontario Confederation of University Faculty Associations; 2019. https://ocufa.on.ca/assets/SQCT-companion-FA-agreements.pdf |
Survey of Ontario Faculty agreements and what they say about SETs / Peer Evaluation of teaching. |
VIDEO |
|
Validity and Equity Problems in Law School Teaching Evaluations, Faculty Conferences: Northwestern Pritzker School of Law.; 2022. Accessed June 12, 2022. https://www.law.northwestern.edu/research-faculty/events/conferences/teaching-evaluations/ |
Video of a recent panel discussion on this issue hosted by Northwestern. |
REFORM |
|
Dalhousie University, “Holistic Evaluation of Teaching Policy”, online: Dalhousie University <https://www.dal.ca/dept/university_secretariat/policies/academic/holistic-evaluation-of-teaching-policy.html>. |
Description of an effort to revamp evaluations of teaching to reduce reliance on SETs. |
WHY (how) DO WE DO STUDENT EVALUATIONS? How a selection of universities explain what they are doing when they do SETs.
|
Berkeley Law, “Teaching Evaluation Procedures”, online: Berkeley Law <https://www.law.berkeley.edu/academics/evaluation-procedures/>. |
University of Ottawa, “Evaluation of teaching and courses”, online: University of Ottawa <https://www2.uottawa.ca/about-us/provost/evaluation-teaching-courses>. |
University of Toronto Faculty of Law, “Course Evaluations | Academic Handbook”, online: <https://handbook.law.utoronto.ca/courses/course-evaluations>. |
University of Western Ontario Faculty of Law, “Academic Policies and Procedures”, online: https://law.uwo.ca/current_students/student_services/academic_policies_and_procedures.html |
York University, “Course Evaluations”, online: <https://courseevaluations.yorku.ca/facultyhelp/midcoursetraining/>. |
NOT JUST FOR T&P: A reminder of how students use SETs sometimes…
|
“The Canons Big Book of Course Reviews: 2022-23 Edition”, Canons of Construction (University of Alberta Faculty of Law Student Newspaper) (17 March 2022), online: <https://www.canonsonline.com/2022/03/the-canons-big-book-of-course-reviews-2022-23-edition/>. |
Workshop Report: Incorporating Disability into the Law School Curriculum
CALT is really grateful to this amazing group of professors who brought us a wonderful workshop on Incorporating disability into the curriculum on June 29 2022.
- David Lepofsky – Disability Advocate, Adjunct Professor of Law, University of Toronto and Chair of the Accessibility for Ontarians with Disabilities Act Alliance
- Laverne Jacobs – Professor, Faculty of Law, University of Windsor and lead author and General Editor Law and Disability in Canada: Cases and Materials. Lexis Nexis 2021.
Co Authors
- Odelia Bay – PhD Student, Osgoode Hall Law School, York University
- Ruby Dhand – Associate Professor, Faculty of Law, Thompson Rivers University
- David Ireland – Associate Professor, Faculty of Law, University of Manitoba
- Richard Jochelson – Dean, Faculty of Law, University of Manitoba
- Freya Kodar – Associate Professor, Faculty of Law, University of Victoria
Anna Lund, Associate Professor, Faculty of Law, University of Alberta (moderator)
David Lepofsky opened the session, describing his report on this topic, originally written for Osgoode Hall Law School and soon to be published in the Windsor Yearbook of Access to Justice as a paper. David noted that he’s given talks at many schools on this topic and welcomes requests to do so – just reach out.
The lead author of Law and Disability: Cases and Materials, Laverne Jacobs (also the 2022 winner of CALT’s Academic Excellence Award!) spoke about the aims and scope of book (available here, tell your librarian and your colleagues), which she noted was “inspired by the notable absence of material about the lives of people with disabilities in law school curricula”. She described it, in part, as a “necessary part of cultural competency of students, disabled and non disabled alike”.
Describing how she brings disability into the public law context, Dr. Jacobs (who teaches, inter alia, Admin law) suggested two practical tips. First, reread an older case in light of changes in the law. She suggested Eldridge v. British Columbia (Attorney General), [1997] 3 S.C.R. 624, with which many will be familiar, which could be reread to ask whether the Medical Commission properly exercised discretion when it did not include sign language translation on the list of funded services. Next, she suggested that a more recent case like Vavilov could be read with a disability lens. Students can be asked to outline the actual impact on people with disabilities, people who access many statutory regimes of benefit provision, for instance. She also recommended, in the area of equality and human rights law, Disability Rights Coalition v. Nova Scotia (Attorney General), 2021 NSCA 70 (CanLII), <https://canlii.ca/t/jjg28> , noting that it focuses on evidentiary requirements which are often important in disability related cases.
Odelia Bay, a sessional instructor in labour law and PhD candidate at Osgoode Hall Law School talked about how she uses her classroom to model accessibility and contrast it with accommodation, working in her teaching to assume that all her students are people with disabilities. She also establishes the difference between social and individual/medical models of disability and how these produce different legal analyses and outcomes. She noted in particular the complexities of episodic disabilities, cases which challenge the idea of predictability in disability and “accommodation”, pointing to some a case she takes up in her contribution to the volume which involves the return to work of a person with bipolar disorder and the labour arbitration over what accommodations were required. She noted that cases also offer the possibility of a discussion of social class and private support. In the case she takes up, the person making the claim was a professor, and the final outcome included insurance payouts which lessened the financial cost of accommodation for the university. But in many cases, people with disabilities will not have this kind of support.
Freya Kodar teaches in the areas of income support law, pension law, social welfare law and tort law. Her engagement with disability analysis focuses on the ways that income support law provides income support across life course, but raises many questions about the adequacy of that support. She particularly noted her efforts to ensure that students are able to problematize the need (in legal contexts) to present disability in a negative light in order to qualify for support. Students can work on identifying the ways that this fits with the medical model and differs profoundly from how a disability justice approach or a social model/critical model might frame the issue.
David Ireland and Richard Jochelson wrote about disability in the criminal law context, considering in particular jury representations for people with disabilities (pointing out that in the case of Indigenous people in Canada, the percentage of people identifying as disabled seems to be higher and perhaps substantially higher than the 1 in 5 usually cited for the rest of the Canadian population) and then the work done at inquests into deaths in state “care”, considering not only deaths in custody, but a other contexts as well. Their contribution also considers how mental and physical disability plays a role (or does not) in sentencing decisions. Like many of the presenters, these two emphasized the need to alert students to the failings of the legal profession in its own approaches to access and accommodation.
Finally, Ruby Dhand, who teaches mental health law spoke about her work in teaching this substantive content of mental health law (which, as she pointed out, intersects with a huge number of overlapping legal areas including human rights law, clinical legal education, health law, criminal law) as well as in furthering discussions about and importance of mental health in the profession. Prof Dhand pointed to the importance of encouraging students to use a trauma informed lens in their own work as lawyers.
This discussion and this book highlight the ways that “disability” issues pervade our law, and people with disabilities are users of all of the doctrine and systems that are used by the non-disabled – with some systems uniquely focused on people with disabilities. Thus the necessity of including this material and habituating our students to thinking about the way that disability is and should be treated in law seems clear. Equally, we are teaching in spaces that include people with disabilities and thus must think about our own practices with respect to disability and access. Finally our students, whether living with disabilities or not and whether planning to practice something that we might label “disability law” or not, must be prepared to have people with disabilities as their clients and colleagues and to respond in professional and appropriate ways to different needs and concern.
CALT is very grateful to all of the presenters in this workshop for their published work and their daily ongoing work to foreground and support the work of access and inclusion in form and practice.
WORKSHOP WEDNESDAY June 29 12:30EST
INCORPORATING LAW AND DISABILITY INTO THE CURRICULUM
June 29 1230-2PM EST
Please register in advance for this meeting: click here. After registering, you will receive a confirmation email containing information about joining the meeting. |
22% of Canadians over the age of 15 have at least one disability. Graduates of law schools will serve clients with disabilities, work alongside colleagues with disabilities and may themselves have or acquire disabilities over the course of their career. Key competencies for graduating law students include being familiar with how the law conceptualizes and addresses disability and having frameworks to critique the shortcomings of the existing law.
The aims of this session are
(1) to provide concrete examples of how topics relevant to meeting the legal needs of individuals with disabilities can be incorporated into a wide range of courses across the law school curriculum and
(2) to engage law professors in a discussion of these topics. Each participant will discuss a different area of law and how they bring awareness of the lived experiences of persons with disabilities into their classroom teaching.
The session will touch on:
- models of disability and theoretical underpinnings
- equality and human rights law
- accessibility legislation (including the federal Accessible Canada Act)
- employment law
- benefits law
- criminal law
- tort law
- administrative law
- mental health law
Time will be reserved after the roundtable for a dialogue among participants and attendees.
The session participants include the contributors to Law and Disability in Canada: Cases and Materials (Toronto: Lexis Nexis Canada, 2021) and David Lepofsky, a longtime disability rights activist, who has a forthcoming article in the Windsor Yearbook on Access to Justice entitled, “People with Disabilities Need Lawyers Too! A Ready-To-Use Plan for Law Schools to Educate Law Students to Effectively Serve the Legal Needs of Clients with Disabilities, As Well As Clients Without Disabilities”.
Anna Lund, Associate Professor, Faculty of Law, University of Alberta (moderator)
- Odelia Bay – PhD Student, Osgoode Hall Law School, York University and co-author Law and Disability in Canada: Cases and Materials
- Ruby Dhand – Associate Professor, Faculty of Law, Thompson Rivers University and co-author Law and Disability in Canada: Cases and Materials.
- David Ireland – Associate Professor, Faculty of Law, University of Manitoba and co-author Law and Disability in Canada: Cases and Materials.
- Laverne Jacobs – Professor, Faculty of Law, University of Windsor and lead author and General Editor, Law and Disability in Canada: Cases and Materials.
- Richard Jochelson – Dean, Faculty of Law, University of Manitoba and co-author Law and Disability in Canada: Cases and Materials
- Freya Kodar – Associate Professor, Faculty of Law, University of Victoria and co-author Law and Disability in Canada: Cases and Materials.
- David Lepofsky – Disability Advocate, Adjunct Professor of Law, University of Toronto and Chair of the Accessibility for Ontarians with Disabilities Act Alliance.