Tag Archives: SoTL

STLHE 2024

A happy update… We won the SoTL Canada Poster Award for this work! Many thanks to the adjudicators at SoTL Canada, as well as to UBC’s Institute for the Scholarship of Teaching and Learning (SoTL Seed Fund), and the UBC Office of the Provost and VP Academic (SoTL Dissemination Fund) for supporting this work. 

I’ll be presenting the following poster at the STLHE 2024 conference.

How can a Longstanding Norm-Referenced Grading Policy Impact Teaching and Learning? Leveraging Qualitative Research for Culture Change in a Quantitative Department

Authors: Catherine D. Rawn (Psychology), Martin Dammert (Dept of Human Development, Learning, & Culture), & Jeanie Woo (Cognitive Systems Program)

STLHE 2024 Poster Rawn Dammert Woo (pdf)

Abstract

Grading student work is required at most universities. Literature has promoted criterion-referenced approaches for decades. Recent innovations in ungrading have centred student self-evaluation based on criteria. Yet norm-referenced grading approaches can persist in some departmental microcultures experiencing pressures of large, under-resourced, multi-section courses, and concerns about preventing grade inflation (c.f., Schinske & Tanner, 2014). Efforts promoting equity, inclusion, and Indigeneity create an imperative to question the use of norm-referenced approaches, even under these pressures (Hogan & Sathy, 2022). How do members of a large quantitative psychology department perceive a longstanding norm-referenced grading policy, and its impact on their teaching and learning?

This poster invites viewers to choose any of three ways to engage with this topic: SoTL, Educational Leadership, or professional growth. 1) The first column summarizes a mixed-methods study designed to answer the question above. Qualitative interview data, analyzed collaboratively using Reflexive Thematic Analysis, highlighted positive and negative lived experiences of faculty, graduate Teaching Assistants, and undergraduates in relation to this policy, offering a clear a mandate for reform.  2) The second column outlines the ongoing process of departmental change, as informed by scholarship on departmental teaching cultures (Roxå & Mårtensson, 2015) and Students as Partners (Felten et al., 2019). The goal of this change is to create a policy where, in theory, for every student can succeed—while facilitating high standards, cross-section equity, and support for course instructors under limited resources. 3) The third column highlights the process of personal and professional development experienced by the lead author, including epistemological growth. Aligning with scholarly analysis of SoTL as boundary crossing transformation (Kensington-Miller et al., 2021), engaging in this project changed this quantitative psychologist’s perspective on her own discipline. Viewers are invited to steer our conversation to learn about this grading policy research, department change, and/or growth.

References

Kensington-Miller, B., Webb, A. S., Gansemer-Topf, A., Lewis, H., Luu, J., Maheux-Pelletier, G., & Hofmann, A. K. (2021). Brokering boundary crossings through the SoTL landscape of practice.Teaching & Learning Inquiry, 9(1), 365-380. Retrieved from https://www.proquest.com/scholarly-journals/brokering-boundary-crossings-through-sotl/docview/2538450692/se-2

Roxå, T., & Mårtensson, K. (2015). Microcultures and informal learning: A heuristic guiding analysis of conditions for informal learning in local higher education workplaces. International Journal for Academic Development, 20, 193-205. DOI: 10.1080/1360144X.2015.1029929

Schinske, J., & Tanner, K. (2014). Teaching more by grading less (or differently). CBE—Life Sciences Education, 13, 159-166. DOI: 10.1187/cbe.cbe-14-03-0054

See also

Lok, B., McNaught, C., & Young, K. (2016). Criterion-referenced and norm-referenced assessments: Compatibility and complementarity. Assessment & Evaluation in Higher Education, 41, 450-465. DOI: 10.1080/02602938.2015.1022136

Felten, P., Abbot, S., Kirkwood, J., Long, A., Lubicz-Nawrocka, T., Mercer-Mapstone, L., & Verwoord, R. (2019). Reimagining the place of students in academic development. International Journal for Academic Development, 24, 192-203. DOI: 10.1080/1360144X.2019.1594235

Hogan, K.A., & Sathy, V. (2022). Inclusive TeachingStrategies for Promoting Equity in the College Classroom. Morgantown: West Virginia University Press.

Officially on Sabbatical!

As of 1 July 2016 I’m officially on Sabbatical! Instead of heading to the classroom this Fall, I’ll be on an extended summer until September 2017. Sabbatical is an amazing opportunity to spend a year working on big picture projects and deep thinking that don’t fit well in the hectic pace of the regular teaching terms. It’s also a chance to catch up on sleep, well-being, time with family and friends, and some travel.

Some projects I’ll be working on include a few papers to submit for publication to journals (3 of which already partially exist but need deep work), the International Program for the Scholarship of Educational Leadership: UBC Certificate on Curriculum and Pedagogy in Higher Education (http://international.educ.ubc.ca/soel/), overhauling my Psyc 101 and 102 courses, continuing to work on curriculum renewal for the BA Psychology degree, and a few other things here and there. I’m working on developing habits to keep me productive enough on these projects while also spending lots of time resting and re-energizing.

If you’re trying to reach me during this time, I’m generally going to be pretty terrible on email. I really hate email. It saps my life energy, which means it cannot be a priority for me during this sabbatical time. If you really need to reach me urgently, try a Tweet (@cdrawn) to grab my attention.

2013/2014 Student Evaluations Response Part 1/4: Intro Psych

Thank you to each of my students who took the time to complete a student evaluation of teaching this year. I value hearing from each of you, and every year your feedback helps me to become a better teacher. As I explained here, I’m writing reflections on the qualitative and quantitative feedback I received from each of my courses.

 

After teaching students intro psych as a 6-credit full-year course for the past three years, in 2013/2014 I was required to transform it into 101 and 102. Broadly speaking, the Term1/Term2 division from the 6-credit course stays the same, but turning Term 2 of Psyc 100 into a semi-standalone Psyc 102 proved more challenging than converting Term 1 into 101. Because these two courses really still form one unit in my mind, and I structure the courses extremely similarly, I will discuss them in tandem.

Across both courses, quantitative feedback was similar (albeit a bit higher in 101 than 102). Students rated the textbook equally high (4.2 & 4.3/5), which makes sense because I use the same text for both, and many students have told me informally that they enjoy reading the book (some qualify this endorsement with “for a textbook”). Check out my overall UMI scores from this year and all previous years here (click to enlarge, and click here for further discussion of UMIs):

IntroPsycHistoricUMIs.2009.2014

 

In the qualitative feedback, many of the same positive and helpful features were highlighted by students in both courses. Overall, students report enjoying and finding valuable the clickers, classroom discussions (often tied to clicker questions), films, opportunities to apply what is being learned, the 3-midterm format that helps stay on top of things even if it’s slightly annoying to be so frequently tested, music before class, the organization of class periods, my enthusiasm and energy, my effort to learn many students’ names, and the Invitational Office Hours. Capturing many of these commonly-mentioned features, one student from Psyc 102 wrote,

It was incredibly impressive how she tried to learn the name of every single student that she interacted with, despite the size of the class. The IOH were also a surprisingly fun experience. The class was very interactive, which definitely helped me learn, and even though I was unhappy about having three midterms at first I think I have to conclude it made studying for each one much easier and less stressful.”

One new element I added to both courses this year were five mini-papers which I called “Writing to Learn” (W2L) assignments (replacing a single 500 word paper I used to assign to be marked by TAs; see last year’s reflection for rationale). Students picked a topic from each of the two chapters about to be tested, wrote a paragraph explaining and applying the concept to their lives, then read 5 peers’ papers and gave feedback to them using peerScholar software. Students received feedback from their peers, and were able to choose any two topics to improve and reproduce on the final exam (no notes!). Overall, students reported finding the Writing to Learn assignments helpful for learning, and some mentioned that reading others’ work was helpful as well (both of these results are consistent with past research on similar writing assignments and peer review). My TAs have reported being able to grasp whether students knew what they were talking about from the writing section on the exams – and my test scores were higher than in previous years, so the goal of increasing learning seems to have been met! However, of the students who mentioned the W2L assignments, many noted that quality of peer feedback received was low. Dr. Peter Graf and I are just starting a project to deal with this very issue of enhancing peer feedback. It may take a couple of years to figure it out in a way that’s scalable to 300-400 students at a time, but we’re working on it.

Interestingly, a couple of students in each course noted my responses to student incivility. In one case it was failing to follow instructions to complete the bubbles during the exam time given, and another case (mentioned a few times in 102, actually), was my response to students talking in class. Side chatter is really only a problem in my first year courses – and it’s a consistent one that varies in severity year-to-year with different cohorts. Interestingly, Gillian Sandstrom and I have a paper about to come out in Teaching of Psychology showing that some chatter is a good thing: it can build a sense of community in a large class. But it can feel disrespectful and distracting to me. Perhaps I should consider building in even more opportunities for structured conversation, because clearly it’s going to happen anyway.

In 102 this year, rather than devoting a whole week to Chapter 2 Research Methods – which I do in 101 in the same place I did back when I taught the 6-credit version – I decided to split it up and cover topics as they came up throughout the term. For example, I used intelligence testing as a chance to discuss measurement and survey designs, and social psychology (specifically Milgram’s studies) as a chance to address the ethics of deception in research designs. Although I think this was a solid solution in theory, in practice there were definitely times when I felt like I was awkwardly wedging topics in to 102. Indeed, a few students mentioned this flow problem too – and it seems to be students who took both 101 and 102 with me who noticed the difference. Hopefully next year I’ll be able to smooth topics out a bit more effectively, perhaps cutting even more material to make more room for these new topics and ensuing discussions.

Although I still would prefer to teach intro psych as a unified whole with the same students over the whole year, apparently that’s not an option any longer. I have begun the process of converting this course into two halves effectively, and given the feedback above, I think I’m heading in that direction.

My Fall 2013 Syllabi

Check out my Fall 2013 syllabi! After spending a lot of time thinking about these courses this summer, I’m excited to share my new syllabi. Psyc 217 features heavily revised Course Goals, new supplemental readings, and a References section listing research that I used to make decisions about this course (e.g., design, policies). Psyc 101 features revised Course Goals and new regular small writing assignments to replace a paper I assigned in years past. I also developed a graphic to help explain the new short writing assignment process.

Psyc 101 Section 005

  • MWF 12-1pm in CIRS 1250
  • 360 students
  • Teaching Assistants: Sara Knauft and Stef Bourrier

Psyc 217 Research Methods Sections 001 and 002

  • MWF 9-10am (Section 1) and 10-11am (Section 2)
  • 92 students per section
  • Teaching Fellows: Allison Brennan, Julia Kam, Jennifer Lay, and Eleni Nasiopolous

See you at Orientations and Imagine on Tuesday, and in class on Wednesday!

Ethics in Teaching

This morning I gave a guest lesson in an “Ethics and Professional Issues” seminar for clinical graduate students. Although I encounter ethical issues in teaching routinely, preparing for this guest lesson gave me a chance to deliberately examine all the activities involved in teaching using the lens of ethics. My thinking was greatly informed by the edited volume called Teaching Ethically that came out last year, which also prompted me to examine the APA code of ethics.

I came up with a list of domains in teaching where ethical issues pop up. You might be surprised by some of these, but not by others. Do you have anything to add?

  • Competency
    • Content knowledge (what to teach)
    • Pedagogical knowledge (how best to teach it; evidence-based assessment)
    • Adequate preparation (class, course)
    • Classroom management (e.g., strategies for dealing with sensitive issues like mental illness, gender differences, ugly history of IQ scores)
    • Self-assess “boundaries of competence” and don’t work beyond them (if must, obtain training)
    • Professional development
    • Seeking advice from and collaborating with colleagues to improve learning
    • Continual improvement (self, course, program, degree)
    • Scholarly teaching/SoTL: using new method without adequate research or preparation
  • Fairness
    • Diversity and inclusivity (access to materials including cost of the textbook, self-disclosure in class activities)
    • Assigning a textbook you don’t use (much)
    • Discourage and pursue cases of plagiarism, cheating
    • Textbook cost
    • Grading and evaluation is clear, competently done, consistent
    • Doing the SoTL work including withholding treatment to one group
    • Accommodations for special circumstances: extra grade chances, re-grades, make up a missed exam
    • Accurately describe your course, set up appropriate expectations (e.g., grades)
  • Faculty-student relationships
    • Trust and power
    • Authorship with graduate and undergraduate student collaborators
    • Writing reference letters for students you know have slim chances of getting in somewhere or who you don’t know that well (and not writing letters when you can)
    • Avoiding multiple (conflicting) roles: research supervisor, employer, teacher, mentor, evaluator, researcher (SoTL), TA/teaching supervisor
    • Consider the impact of challenging students’ core beliefs (e.g., God, evolution, trust in authority, gay rights…)
    • Social media: Facebook friends, Twitter following à blurring edge of professional relationship
    • Assigning a textbook when you’re and author and will receive royalties
    • Accepting gifts from students (and textbook publishers, for that matter)
  • Confidentiality
    • Grades files shared over email (security)
    • Sending grades over email to students using non-UBC email addresses
    • Storage and confidential shredding of paper material
    • Non-disclosure to parents
  • Law
    • Understanding copyright laws (including digital copyright)
    • Complying with copyright laws