Comparing 2021 to my fully renewed 2023 and adjusted 2024 offerings, how do students receive the course?
Are there different strengths and suggestions before and after the renewal? Overall, is the renewal doing what I hope it is doing? That is, strengthening students’ critical understanding of our discipline’s dominant ways of knowing, while developing their (researcher) identities?
This post continues from Learning from my previous Students PSYC_V 217 Research Methods.
Since 2010 I have regularly taught students in this course at 9am and 10am on MWF in Term 1. This graph presents UMI interpolated medians starting from W2021, when we were back in person classes but masking. I didn’t teach in W2022, and when I returned for W2023 I felt compelled to overhaul this longstanding course (see my poster for STLHE 2024 for some rationale; all syllabi are available on my Courses page). As you can see in the graph below (click to enlarge), quantitative results suggest a slight drop in some items since the course renewal, but levels remain acceptable (in my opinion). Academic year 2021/22 was an unusual year in many ways: we were all relieved (also anxious) to be back in class after a year of learning fully online. As a result, those numbers may have been unusually high. Unfortunately, the item wording changed a fair bit in 2021, so we can’t compare with PSYC 217 SET W2018-W2020 to help illuminate.
So what had changed in W2023? The talk I gave at Improving University Teaching Conference 2025 provides a pretty good overview. My relationship to this course content had changed, because who I am and how I see the world had changed. I had been learning Indigenous ways of knowing and re-learning history (e.g., Indigenous Canada course), and understanding privilege and racism and inclusion in new and deeper ways–personally as a human, in the classroom, and in our discipline of psychology. I had begun using qualitative research methods to challenge our grading policy, and this work had challenged my assumptions about validity and reliability and truth and evidence… all concepts at the core of what I’d been teaching in Research Methods for 15 years. I had also spent months ministering to my Gran–my primary caregiver throughout my life–through acute illness, her death, and complicated aftermath. And of course, the pandemic had already challenged my assumptions about what teaching looks like, what content was actually essential, and what really matters by the end of a course: that students feel seen and guided in their growth. I could no longer stand with integrity and say, come join us in doing postpositivist quantitative research, and test on tiny details, without engaging a critical examination and featuring identity. My identity shift had shown me first hand how much researcher identity matters to our research. To teach Research Methods without embedding identity, positionality, and critical interrogation of postpositivism felt like a lie. I threw out almost everything. Compare my syllabi from W2021 to W2024 (and W2019 to see the impact of the pandemic).
I shifted content to enable sampling some concepts from Indigenous research paradigms, qualitative methods, and current conversations regarding EDI challenges within the field of psychology. Students still receive basic training in quantitative research methods, but the lenses through which we gaze at those methods are different. Two new Course Learning Outcomes point to a couple of major changes:
- (4) Engage with a variety of published journal articles to discuss examples of concepts, practices, and values core to conducting and writing about psychological research. From here, I created a new method of testing in which students are given a journal article and asked questions about it (which led to revised class activities to focus on developing those skills). See comments in Learning from my previous Students PSYC_V 217 Research Methods showing these tests were perceived as challenging but worthwhile, and could be supported even more.
- (8) Reflect on your own positionality, your values and perspectives, and how they might influence how you engage with psychology’s research processes. Discuss what skills, methods, values, perspectives you want to take with you beyond this course, which to leave behind, and which you might continue to try out or perhaps continue to question. I developed a whole series of “Researcher Notes” (written in class by hand) to scaffold everyone’s development at least a little. For those who wanted to dive deeper, I developed an optional “Researcher Identity Integration” paper. See my IUT talk slides Cultivating Researcher Identity and Importance of Positionality among Students in Quantitative Psychology Research Methods for an overview and evaluation. (My research analyzing the Researcher Identity Integration papers is currently under review.)
What do students’ qualitative feedback indicate? I asked Microsoft Copilot for assistance analyzing this very large dataset (about 30 pages of small font comments from W21, W23, W24). Here’s the question I used: “I substantially overhauled PSYC 217 in 2023. Here are the comments from PSYC 217 the year before that change (i.e., 2021). What similarities and differences do you notice in how students comment on this course? How would you evaluate the nature of the course shift in PSYC 217?” Here’s some of the answer, verbatim:
“This is a fascinating comparison because, reading 2021 against 2023–2024, I don’t see a complete replacement of one course with another. Instead, I see a continuity in teaching philosophy but a major shift in what students perceive themselves as learning and becoming. The 2023 redesign appears to have shifted PSYC 217 from being primarily a well-taught research methods course into an identity-building and perspective-transforming course about becoming a researcher. … The course shifted from teaching students how psychological research works to helping students understand themselves as participants in psychological knowledge-making. That doesn’t mean the original course was weak—the 2021 comments are actually very positive. Rather, the redesign appears to have added a substantial layer of personal, intellectual, and professional development on top of an already strong research-methods foundation.
If I had to characterize the redesign in a single sentence: The course shifted from teaching students how psychological research works to helping students understand themselves as participants in psychological knowledge-making.“
Here’s some more, including student quotes:
The Biggest Difference: From Skills to Identity The strongest shift I notice is in the level of identity development students describe.
2021 Students frequently said things like:
- “I learned research methods.”
- “I understand psychology research better.”
- “The project helped apply concepts.”
- “I could see whether research was something I might pursue.”
Research is still largely external. Students are learning about research.
2023–2024 Students frequently say things like:
- “I grew as a researcher.”
- “Experiencing psychology as a psychologist.”
- “Helped me understand my place in psychology.”
- “I can see myself as part of knowledge creation.”
- “Researcher identity.”
- “Positionality.”
- “Who I am as a researcher.”
The language becomes much more personal and developmental. Research is no longer external. Students increasingly describe themselves as participants. If I were coding these datasets qualitatively, I would almost certainly create a new thematic category called identity development that would barely register in 2021 and become a major theme in 2023–2024.
WOW. Remember, Copilot didn’t have access to my syllabi. I didn’t tell it how I transformed the course, only that I had done so. This analysis was only from what students wrote in their end-of-term evaluations. I’m blown away by how much student comments, as interpreted by Copilot, reflect my intentions to help students develop their identities through this research process. If I had come to these conclusions myself I would have doubted how pronounced they were. I would have worried about me seeing what I want to see, what I expected to see, rather than what was actually there.
I ended my Copilot dialogue with this question: “Are there any risks or cautions or caveats you would offer about your own analysis?” It offered many useful notes, many of which align well with what I teach in Research Methods! 🙂 Of key relevance here:
“The most compelling finding isn’t “students liked me.” If I were worried about bias in my own analysis, this is actually where I would have expected it to creep in. But what surprised me was that the most interesting pattern wasn’t praise. It was the change in what students talk about. In 2021, students largely described: learning research methods, doing projects, flexibility, organization. In 2023–2024, students increasingly described: researcher identity, positionality, belonging, self-reflection, their place in psychology. That shift is visible even if you completely ignore the positive tone of the comments. To me, that makes the redesign story particularly persuasive.
The statement I would feel most confident defending is: … “Following the redesign of PSYC 217, students increasingly articulated changes in how they understood themselves in relation to research, knowledge production, and the discipline of psychology, suggesting a shift from primarily skills-focused learning toward more identity-oriented and transformative forms of learning.” That’s a strong claim. And importantly, I think it’s also a defensible one.”
I was hoping to answer: Overall, is the renewal doing what I hope it is doing? That is, strengthening students’ critical understanding of our discipline’s dominant ways of knowing, while developing their (researcher) identities? Yes. There appears to be solid evidence that this renewed course is indeed working as intended… at least for the subset of students who responded to the comments section in the course evaluations.
I also was hoping to answer: Are there different strengths and suggestions before and after the renewal? “The biggest difference” is already answered above (skills to identity), but there are additional details offered too. Positionality appears in W23 after never appearing in W21. Critical thinking is discussed prior in terms of applying concepts and analyzing studies, whereas post-renewal involves critiquing the discipline itself and considering alternative perspectives. Belonging shifts from caring and welcoming classroom environment, to feeling a sense of belonging in psychology and academia and being seen. These all resonate with my intentions. Complaints in W21 feature too many assignments and Canvas complexity, shifting in W23-24 to understanding expectations for article tests including wanting model answers. I have previously discussed my plan for addressing this new need.
It’s a little creepy how much Copilot sounds like a person. But overall, this experience using Copilot to help me with my analysis has been tremendously helpful.