Category Archives: from the classroom

2014/2015 Student Evaluations Response Part 2: Psyc 101

Thank you to each of my students who took the time to complete a student evaluation of teaching this year. I value hearing from each of you, and every year your feedback helps me to become a better teacher. As I explained here, I’m writing reflections on the qualitative and quantitative feedback I received from each of my courses.

 

After teaching students intro psych as a 6-credit full-year course for three years, in 2013/2014 I was required to transform it into 101 and 102. Broadly speaking, the Term1/Term2 division from the 6-credit course stays the same, but there are some key changes because students can take these courses in either order from any professor (who uses any textbook). These two courses really still form one unit in my mind so I structure the courses extremely similarly. I have summarized the quantitative student evaluations in tandem. Students rate my teaching in these courses very similarly. However, I will discuss them separately this year because of some process-based changes I made in 102 relative to 101.

IntroPsycHistoricUMIs.LastFiveYears

To understand the qualitative comments in Psyc 101, I sorted all 123 of them into broad categories of positive, negative, and suggestions, and three major themes emerged: enthusiasm/energy/engagement, tests/exams, and writing assignments. Many comments covered more than one theme, but I tried to pick out the major topic of the comment while sorting. Twenty-one comments (18%) focused on the positive energy and enthusiasm I brought to the classroom – enthusiasm for students, teaching, and the discipline (one other comment didn’t find this aspect of my teaching helpful). All nine comments about the examples I gave indicated they were helpful for learning.

Thirty-four comments focused largely on tests (28%). Last year I implemented the two-stage tests in most of my classes, and I expected to see this format discussed in these evaluations. All but one of the five comments about the two-stage tests were positive. Other positive comments mentioned that tests were fair and that having three midterms kept people on track. Yet, the major theme was difficulty. Of the 17 negative comments that focused largely on tests (representing 14% of total comments), the two biggest issues were difficultly and short length (which meant missing a few questions had a larger impact). The biggest theme of the suggestions (5 comments) requested practice questions for the exams. There is a study guide that students can use that accompanies the text. I wonder how many students make use of that resource? Do they know? Would that help students get a better sense of the exam difficulty? This is a major question I want to ask my upcoming Student Management Team (SMT) this year. Still, it’s important to keep in mind that these comments represented a minority in the context of all 123 that were given.

Twenty-five comments (20%) focused largely on the low-stakes writing-to-learn assessments. Five times throughout the term students write a couple of paragraphs explaining a concept and then applying it to understand something in their lives. They are each worth 2%. When I implemented this method two years ago I also added a peer assessment component, such that 1% has been completion, and 1% comes from the average of their peers’ ratings of their work. In year one I used Pearson’s peerScholar application, and in year two (2014/2015) I switched to the “Self and Peer Evaluation” feature in Connect (UBC’s name for Blackboard LMS)… which was disastrous from the data side of things (e.g., giving a zero instead of missing data… and then counting the zero when auto-calculating the student’s average peer rating!). As I expected, most of the comments about the assignments were about peer review failures: missing comments, missing data, distrust of peers as being able to rate each other, distrust at only receiving 3 ratings that can be wildly different sometimes, difficult to get a good mark, suspicion that peers weren’t taking it seriously. For 2015/2016, I have made three major changes to help improve this aspect of the course:

  1. Return to the peerScholar platform instead of Connect, which should fix part of the missing data problem (and it can now integrate with my class list better than it did before),
  2. Review 4 or maybe 5 peers rather than 3,
  3. Implement a training workshop! With the help of a TLEF, Peter Graf and I have been developing an online training module for our courses to help students learn to use our respective peer assessment rubrics and test out the rubric on some sample essays. Our main hypothesis is that students will be more effective peer reviewers—and will feel like peers are more effective reviewers—as a result of going through this process. More to come!

As can be seen in the graph above, quantitative ratings of this course haven’t changed too much over the past few years. The written comments are useful for highlighting some areas for further development. I look forward to recruiting a Student Management Team this year to make this course an even more effective learning experience for students!

2014/2015 Student Evaluations Response Part 1: Psyc 217

Thank you to each of my students who took the time to complete a student evaluation of teaching this year. I value hearing from each of you, and every year your feedback helps me to become a better teacher. As I explained here, I’m writing reflections on the qualitative and quantitative feedback I received from each of my courses.

 

Research Methods is the course I have taught more than any other: since 2008 I have taught 1022 students across 14 sections. I am also the course coordinator, which means I help new research methods instructors prepare for their sections including ordering textbooks, I organize the labs including TFs and room bookings, and I facilitate the poster session. And I write the Canadian edition of the text we use (2nd edition forthcoming February 2016). I spend a lot of time thinking about this course!

Over the years I have incorporated many suggestions initiated by students in student evaluations, and have made changes to the course design based on student evaluation feedback. In 2014/2015, I implemented one major change to the course: I converted my three tests and the final exam to a two-stage format. Students write the test first on their own, then break into groups to write it again (see my blog post about this method, with resources). The largest challenge I knew I faced going in was timing, particularly for the tests: Our class periods are just 50 minutes long, and my tests were already notoriously lengthy. It was with this lens that I approached reading my student evaluations for this past year. Do I keep the two-stage tests?

I examined the quantitative data first. As is clear from the graph, there were no major differences relative to previous years. Notably, The fairness of the instructors’ assessments of learning item was rated higher than usual, though that difference was small. No indication of a disaster. Yay!

Psyc217historicUMIs.LastFiveYears

Next, I evaluated the qualitative data. As I sorted into positive and negative columns, two topic themes seemed to be emerging: tests and enthusiasm. As in past years, students appreciated the energy and enthusiasm I bring to the classroom (especially at 9am and especially with this topic). Out of 128 comments, 29 of them (23%) specifically mentioned energy or enthusiasm (with just a couple of those recommending I tone it down a bit).

Coincidentally, the same proportion of comments (29, 23%) mentioned the tests in some way. Six comments endorsed the three (rather than two) test format, indicating it helped them keep on top of studying, although three comments mentioned that tests 2 and 3 were too close together, and another three indicated they would have preferred two tests. Seven comments mentioned enjoying the two-stage format were positive, indicating that it provides opportunities to work together, make friends, and receive immediate feedback. The two negative comments that specifically mentioned the two-stage format did not disagree with it per se, but felt that this format exacerbated a different problem: feeling rushed. Seven comments specifically mentioned feeling rushed during exams. Two others indicated that the fix implemented for test #2 worked well to address the timing issue. Still, it seems that timing during tests was the clearest challenge in my course. Despite my best efforts to shrink the tests, there is a small group of students reporting they remain too long for the required tasks. I’ll consider strategies for preparing students for this pace.

Overall, the two-stage tests seemed to work well for most students, and grades were still within the acceptable range for our department. I enjoyed giving exams much more than I used to, and I was able to relax and hear the conversations students were having as they debated correct answers. Anecdotally, I was able to witness deeper learning and (mostly) positive group dynamics during the second stage (luckily I have other peoples’ data to offer as evidence that it works to promote learning!). Two-stage exams: you’re staying!

Using Social Media to Build a Class on Social Media

Over the next year I’ll be developing a course called the Psychology of Social Media, which I will teach as Psyc 325 in January 2016 at UBC. This course is currently listed as a developmental course, but we will emphasize themes of social and personality psychology (which relate to identity and personality development). I’m excited to be developing this new, rich course, and have already begun brainstorming.

While I’m at a conference next week (Society for Personality and Social Psychology in Long Beach, California), I’m hoping to gather some resources by attending relevant sessions as well as engage with the crowd itself using social media (I know… meta!). I have created the following GoogleDoc to help me (and anyone else who finds it useful) build a set of readings and other resources for a Psychology of Social Media course.

Have ideas? Post them here! Want ideas? Gather them here!

https://docs.google.com/document/d/1l5nKrzhc9p914RwVyBhKRrAX2kwZX3YWtURuV_Y-QFw/edit?usp=sharing

(Or the shortened link from Hootsuite: http://ow.ly/J9gLk)

On Using Clickers

I was asked this week to reflect on why I use clickers for use on the Faculty of Arts ISIT website. Here are my responses…

  1. Background information about your course
    • I use clickers in four of my courses: Introductory psychology (Psyc 101 and 102; 260-370 students), Research methods (Psyc 217; 90 students but as few as 30 years ago), Statistics (Psyc 218; 90 students).
  2. How did you use iClickers in your course and what made you decide to do this?
    • I use clickers every class period for numerous purposes. I ask multiple choice questions to test whether students understand a previous concept before moving on to something new, to spark discussion (especially when there is no one definitive right answer), to survey attitudes about the topic or course (like where to put the laptop-free zone), and as a timer for groupwork or peer-to-peer discussions so we all know how long they have been working. Plus the instructor clicker advances my slides and is easy to use.
  3. What has been the result?
    • When I’m using clickers most effectively, it means the students and I are in dialogue throughout the lesson. Students are constantly developing greater insight into whether they understand the concepts—and so am I (see also research on the testing effect, for evidence that testing rather than just reviewing contributes to longer lasting learning). It’s also engaging and motivating: Students want to know what the answer is, and will sometimes even cheer for correct responses. Questions give us a point of discussion that regularly takes us all deeper than just a surface understanding, as students may argue for one or another answer and we have to unpack why. Also, I have data showing that talking in class is related to feeling a sense of community. Clickers help me trigger and steer those conversations to build community.
    • Other benefits: I have become better at asking multiple choice questions, because I get immediate (and vocal!) feedback if an item is worded unclearly. Every student participates (not just the brave) – and if someone chooses E for a question that gives options A-D only, I get to smile and lightly invite that person (whoever it is) to make the choice to learn today.
  4. What are some of the challenges you’ve faced and is there anything about your approach you would improve or change?
    • When I started using clickers, the biggest challenge I faced was responding in the moment to whatever response pattern comes up. Over the years I’ve developed a toolkit of options to use, depending on the response pattern, but sometimes I’m still surprised by what comes up! For example, if it’s 50/50 between two options, I often invite a “turn to your neighbour, discuss, and revote” which usually clears up the problem. Other times I don’t reveal the answers immediately to the class (though I can see them on the receiver), but instead ask the whole class why someone might choose a particular response over another, and then reveal.
    • I’ve toyed with the idea of using a service with text-based responses (e.g., TopHat), rather than the i>clicker with just multiple choice options, but haven’t yet. For now, multiple choice questions meet my needs.
  5. Do you have any advice for instructors hoping to implement this in their courses?

Revising my Teaching Philosophy

Although I’ve polished/reconfigured my teaching philosophy a few times over the years, I’m feeling compelled to take it in a new direction. It’s not that my teaching has taken on an entirely new form… well, maybe in some ways it has. I’m feeling a greater sense of clarity these days, specifically around collaboration and community-building. Here is my latest draft of my single sentence nutshell version, followed by lengthier elaboration/musings.

The goal of my teaching practice is to help people develop ways of thinking and doing that are informed by the methods and evidence of quantitative psychology, and that will prepare them to engage in their social worlds throughout their lives.

I chose many of these words very carefully.

goal, my teaching practice: indicate my own process of ongoing development. For me, teaching is not a skill I can check off as “mastered” — which is actually one of the reasons I enjoy it so much. It’s a set of skills and attitudes I can get better at through reflective practice, but it’s never “done.” Also noteworthy at this point: although this statement is grounded in what (I think) I do, it is also somewhat aspirational and intended to help me make decisions about how to change my teaching in the future.

help, develop: In my most effective teaching moments, I am acting as a guide, not some sage trying to fill brains with what I know. I view myself in a supporting role, and that probably has something to do with the most impactful teachers in my life, who treated me as an individual person (see below) and pushed me to develop new skill sets, regardless of how scary or challenging that was (I’m thinking of you Clinton, Ross, Holmes, Biesanz, Vohs…). Because many of my core courses are foundational  (introductory psychology, research methods, statistics, teaching of psychology for graduate students), I sometimes get to see and support developmental change in my students across much of their degrees. When possible, I provide opportunities for students to practice and build on past work.

people: reminds me that my students are more than just students (just as I am more than their instructor). I put this into practice by saying this, right before every exam begins: “and remember, your value as a person has nothing to do with your performance on this or any other exam.” Year after year, students thank me for this tiny reminder that they are not the sum of their grades. Also, this word reminds me that some of my teaching activities reach beyond traditional classroom settings and involve people who aren’t necessarily “students” per se.

ways of thinking and doing: I am increasingly aware of the fact that when I use lecture, I am giving students ample opportunity to practice note-taking and listening… which are not entirely useless skills, but are maybe not the best use of our time together (see above). With each new course I develop and each course I revise, I strive further toward spending class periods giving students time to practice their thinking and doing (asking, answering, brainstorming, designing, summarizing, analyzing, performing, playing, creating). To support doing-driven assignments, I need to work on building space in my course for further skill development.

methods and evidence of quantitative psychology: By training, I am a quantitative social psychologist. I teach research methods and statistics, write a research methods textbook, and deliberately chose to use an introductory psychology textbook that emphasizes research methods and combating pseudoscience. I think the tools and content of psychology are useful for critically consuming information that is thrown at us, and for engaging productively in the world. In my classes, I create opportunities for students to apply concepts and methods (test questions, assignments).  Importantly, I use the methods and evidence of quantitative behavioural sciences to inform my teaching practice.

engage: Here’s a word that was core in my old teaching philosophy. The way I’m looking at it now is this: I structure my learning situations to require learners to participate and engage with the material and each other. I use various common techniques: vivid examples, videos, attention grabbing questions sprinkled throughout class (clickers), projects that mean something (e.g., identify a problem you’re having and find research to help), and some that are uncommon (2-stage exams). I engage students in the learning process by breaking down my 1st and 2nd year courses into four chunks (three midterms and a cumulative final) — and in my intro course I require students to begin studying at least 2 days before each midterm because a short low-stakes concept check paper is due. Another way I try to engage is by building community…

social: Now we get to the primary reason for this whole revision. Most of the changes I’ve made over the years to my courses have been to make them more social. Community-building and collaboration feature heavily in my courses, and I’m ready to commit to them as defining features of my teaching. The research literature supports this approach. “Active learning” techniques that involve some sort of peer-to-peer engagement seem to work best to help students learn. I also believe that in order to engage effectively in their personal lives and broader society, people need practice developing skills and attitudes needed to work with and learn from others. The social looks different in different courses (in part depending on class size and level), but includes two-stage/team tests, group projects, peer evaluation, reviewing peers’ written work and presentations, informal discussions prompted by clicker questions, team writing, occasional field trips, and invitational office hours. Also, the more students talk in class, the stronger sense of community they feel. Because my classes tend to be very large (88 is the smallest, excepting my grad class), and many students have traveled from around the world to be here at UBC, increasing the social in class may help struggling students feel more connected. Next year, my new course on the psychology of social media will explore the theme of technology-mediated connection and will require it!

throughout their lives: my greatest teaching efforts are lost if they do not contribute to some lasting change for some people some of the time. My hope is that some of the skills, tools, ways of thinking and doing and engaging will positively influence the people I teach. If ever-so-subtly.

 

Some other thoughts

Technology-enhanced experience. I use technological tools wherever it makes sense to facilitate my goals. Examples: clickers, Connect, self-and-peer evaluation (Connect, peerScholar), scratch cards and live test grading, turnitin, videos

Integrated Course Design model. With each new course I’m developing, I’m getting better at applying Fink’s model. Basically, strive for alignment among course goals, learning assessments, and in-class experiences.