Category Archives: Reflections

Student Evaluations of Teaching 2012/2013: Part 2 Research Methods

Thank you to each of my students who took the time to complete a student evaluation of teaching this year. I value hearing from each of you, and every year your feedback helps me to become a better teacher. As I explained here, I’m writing reflections on the qualitative and quantitative feedback I received from each of my courses.

Psyc 217: Research Methods

I went into this year especially excited about research methods, as I got to use a textbook with my own name on it! Wow, what a thrill! Perhaps reflecting this extra-potent boost of enthusiasm, my quantitative results were overwhelmingly positive this year. Interestingly, I seem to have connected especially effectively with my 10am section. Ratings from my 9am section were positive too (see the mint green bars on the graph linked above)… and on par with years past. But scores from my 10am section were the highest I’ve ever received (see the light purple bars on the graph)! Because I taught the two sections pretty much the same way, I’m not sure what can account for the difference. Suffice it to say, in my mind it was an especially awesome year… and many of my students seem to have felt that way too.

When I teach research methods, it’s often at 9 and 10 in the morning, and I do my very best every day to bring the energy. For many people, this material isn’t exactly inherently exciting. As one student wrote, “Based on what I’ve heard from friends and acquaintances at UBC, research methods is one of the most disliked courses offered at the university due to its sheer boringness.” Thankfully, this student continued, “that said, this instructor did a phenomenal job of teaching the course in a way that students found the material relevant and exciting (to the extent that this material can be exciting).” Such an assessment is the most common comment coming from my student evaluations in this course: Students expect this material to be dull, but I bring it alive. That’s exactly what I strive to do every single day. I’m satisfied that my well-caffeinated efforts are effective for my students.

A few other topics were noted by small subsets of students. Two topics drew ambivalent assessments: groupwork and in-class activities. People seem to have a love-hate relationship with groupwork. First, only a handful of people mentioned it at all, leading me to suspect that mostly people feel neutrally about it (perhaps recognizing its inherent challenges and strengths). The people who noted liking the team project still found it a lot of work, but recognized the value in it. The people who didn’t seem to work as well with teammates report viewing it as a frustrating waste of time. Each year I hear this dichotomous assessment. One thing I tried out last year in response to one particularly struggling group was a mediation meeting, during which I acted as mediator. It seemed to work well to get that group through effectively to the end of the course. To broaden this service and reach the struggling groups I don’t hear about, I am creating a form-based mediation request process for this year. That may sound like a cold approach, but I’ve given it much thought. After years of imploring people to come to me face-to-face to help solve their group challenges, I note that very few groups—or individuals struggling within groups—ever come to me. By formalizing this process, I hope to remove some of the emotion around “tattling,” and treat it as just another issue that needs to be dealt with, just like a grade change request. Hopefully this new process will help reach those extra few groups who are struggling on their own so they can move forward and perform well in group tasks.

In-class activities make material memorable, illustrate difficult concepts, up the energy and attention levels, and make learning fun. Every year, dozens of students report appreciating them. However, there is a small minority of students who don’t appreciate the time spent on these active learning adventures (yes, that’s the subtitle of my blog… see where I’m headed). I’m committed to student learning, and one of the hallmarks of my teaching philosophy is to get out of the way. And data supports my commitment to using active learning techniques (Armbruster, Patel, Johnson, & Weiss, 2009; Deslauriers, Schelew, & Wieman, 2011; Hake, 1998; Prince, 2004; Ruhl, Hughes, & Schloss, 1987; Yoder & Hochevar, 2005). I encourage people who are considering taking research methods or statistics with me (or any of my courses, really), to be ready to engage actively during class. If you’re not up for having fun while learning, my section might not be for you.

The last topic I’ll touch on is the textbooks. A few students noted how much they found my textbook worthwhile (yay!), with one student going so far as to say “I loved her book she wrote, very clear, informative, concise, probably the best textbook I ever used and read due to how clear it is to understand, with all the learning objectives.” I can’t take full credit for that readability (thanks to Cozby for laying such a strong foundation in his 10 prior editions!), yet I’m glad this text is being perceived as helpful. Unfortunately, the Stanovich text once again was voted unhelpful. The messages are useful, but even I find many examples dated and the chapters too lengthy for the points they make. Two years ago I wrote learning objectives and emphasized “get in, find what you need to know, and get out approach” in an attempt to make Stanovich’s text more accessible. Since then, there have been fewer complaints about Stanovich’s text, but a small, consistent group remain. I’ve been back-and-forth on this text for quite a while now, and I’m strongly considering replacing it with a few key peer reviewed articles/commentaries. I have some deep thinking to do in the coming weeks!

Many thanks to all my Psyc 217 students in 2012/2013 students who completed this evaluation. The response rate this year was 67% across both sections, which is my highest rate ever. And thanks to everyone for a really fun year of learning about research methods!

References

Armbruster, P., Patel, M. Johnson, E., & Weiss, M. (2009). Active learning and student-centred pedagogy improve student attitudes and performance in introductory biology. CBE—Life Sciences Education, 8, 203-213.

Deslauriers, L., Schelew, E., & Wieman, C. (2011). Improved learning in a large-enrollment physics class. Science, 332, 862-864.

Hake, R. (1998). Interactive-engagement vs. traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66, 64-74.

Prince, M. (2004). Does active learning work? A review of the research. Journal of Engineering Education, 93, 223-231.

Ruhl, K., Hughes, C., & Schloss, P. (1987). Using the pause procedure to enhance lecture recall. Teacher Education and Special Education, 10, 14-18.

Yoder, J. D., & Hochevar, C. M. (2005). Encouraging active learning can improve students’ performance on examinations. Teaching of Psychology, 32, 91-95.

Student Evaluations of Teaching 2012/2013: Part 1 Intro Psych

Thank you to each of my students who took the time to complete a student evaluation of teaching this year. I value hearing from each of you, and every year your feedback helps me to become a better teacher. As I explained here, I’m writing reflections on the qualitative and quantitative feedback I received from each of my courses.

Introduction to Psychology

Overall, I was very pleased with my introductory psychology students’ assessments of my teaching this year. Quantitative data were all highly positive (see the orange bars here).  In the qualitative responses, many students highlighted my enthusiasm, organization, strong communication skills, and care for getting to know them as individuals. Because these themes emerge regularly, particularly from my introductory students, I have learned to embrace them and lean into them – a strategy that seems to be working! Specific features of our course/my teaching that many students noted as particularly effective include i>clickers as a way to engage attention and reinforce learning, invitational office hour as a way to connect personally with students, videos and demonstrations in class to make points memorable, and having three midterms (rather than two, as we did in 2nd term) as helpful for keeping on top of readings. For next year, I’ll switch term 2 (which will be a separate course, Psyc 102) to a three-midterm format.

Mid-way through the year, I was discussing the use of Learning Objectives with students at an IOH. During that discussion, someone suggested I keep the Learning Objectives posted somewhere throughout the class, to serve as a reminder of what students need to especially focus on understanding and doing. In response, I committed to posting the LOs before each class period on our Vista course website. That way, students can consult them throughout the class (provided they have a device to do so… which many do). Many students noted that they found this cumulative list of LOs helpful during class as well as later as an exam study tool. There are many reasons why I can’t post my slides before class, but I can commit to posting LOs. Because this simple thing seemed to be so helpful for at least a subset of students, I will continue doing this in the fall (and perhaps extend to all my courses).

Exams and papers were two discussion points that were noted in various ways in quite a few posts. Regarding exams, many students noted they were challenging (which I embrace), yet a few added that they felt unprepared for this level of difficulty. One of the things I will consider doing next year is holding an optional review session outside of class time before at least the first midterm. I’m not willing to simply re-teach material (as if coming to the review session would be enough studying, or would substitute for coming to class thrice weekly), so I’ll have to think more about how to approach them (see Regan Gurung’s Observer article). Logistically this could be tricky, especially if there are many students who attend. I’ll have to give this possibility some more thought.

For the past four years I’ve required students write a 600 word paper each term on one of two or three topics each term. All papers have in common a requirement to do something to apply a course concept, summarize what they did or saw, and explain how that event illustrates that concept (e.g., write a study plan applying principles of memory). A handful of people gave really thoughtful feedback on the main challenge this paper poses: 600 words isn’t long enough to dive deeply into the material. I haven’t been thrilled with the quality of the papers recently… in part because I simply can’t offer a scaffolded process with meaningful feedback to 250-350 students a term. To help address this feedback issue, I turned it over to the students this year. I added a requirement to the paper that people give peer feedback to four of their peers’ papers using peerScholar software, and gave people a week to incorporate the feedback they received (if they chose to do so) before final submission. To my surprise, not a single person mentioned peerScholar in their qualitative feedback. Was it just not memorable? Not helpful? I can’t tell. I recall having called a vote using i>clickers at the end of term 1 during which people endorsed it as useful and wanted to use it again… but it didn’t show up at all in student evals! I’m really not sure what to make of that, but I presume students didn’t hate it or else I’d have heard about it. I’m considering a new approach to the papers, inspired by this ToP article I wrote about a few weeks ago, while incorporating peer ratings through peerScholar as a study tool. I think that could work to satisfy both my writing-to-learn and peer feedback goals.

Notably, the graph highlights the fact that my classes of ~250 (2010/2011, 2012/2013) seem to be rated more highly than my classes of ~350 (2011/2012). Given this pattern, I am a bit nervous heading into next year. For the past three years I have been fortunate to teach some of the last three 6-credit sections of Introductory Psychology (Psyc 100). From now on, admin has decided that all sections will split into 101 and 102. The content mostly mirrors the first half and second half of Psyc 100, respectively, but with two huge differences: it’s not (entirely) the same group of students, and because neither is a prereq for the other, 102 students might not have had 101 at all (let alone with me). Having the same group of students all year has afforded me the rich opportunity to establish relationships across eight months with the same group of students. I can invite every single student to an Invitational Office Hour over that length of time – which has led me to personally meet 70% of my intro students in each of my last three cohorts. In 2013/2014, with 350 students in Psyc 101 and potentially an entirely different crew of 250 students in Psyc 102, there is logistically no way I can invite everyone. Because IOH has been so enormously successful, I will continue it. But it will need to be by random selection (plus an open invite to keen students), and I will no longer be able to offer 1% for coming and “engaging in learning” because I can’t offer that opportunity to everyone. I’ve met such interesting students and established great relationships and community through IOH… I hope people still come!

My advice if you’re choosing intro psych for 2013/2014: sign up for Psyc 101 and 102, in that order… with me J (or someone else, but I’d love to meet you!). Then, if you’re in my sections, come to IOH so we can get to know each other!

Many thanks to all my Psyc 100 students in 2012/2013 students who completed this evaluation. The response rate this year was 62%, which is among my higher rates. And thanks to the whole class for a fun year of learning about psychology!

Coming Soon: My Responses to Student Evaluations of Teaching

Thank you to each of my students who took the time to complete a student evaluation of teaching this year. Overall, your response rate was the highest I’ve ever received! Thank you! (Since 2009/2010, response rates across all courses have been 54%, 55%, 49%, and, this 2012/2013 year, boasted the high of 62%.) I value hearing from each of you, and every year your feedback helps me to become a better teacher.

When I receive my evaluations, I take my time exploring and interpreting both the qualitative and quantitative feedback. I have already created graphs depicting results from the University Module Items, which are 6 questions that are asked about every instructor across campus. I have posted those graphs (and the precise wording of each question) here for your consideration. With respect to the open-ended responses, I appreciate and consider every thoughtful comment. The ones I write about are typically those that reflect common themes echoed by numerous students.

I am in the process of writing reflections on the feedback I received from each of my courses, and will be posting them over the coming weeks. Check out last year’s reflections on intro, research methods, stats, and Psyc 208 special topics), for comparisons.

Reflecting on APS

Last week I attended the Association for Psychological Science Convention for the first time, including the Society for the Teaching of Psychology  (STP) Teaching Preconference. I am grateful to STP for awarding me an Early Career Travel Grant to help me offset the cost of my flight all the way from Vancouver Canada to Washington DC.

One of my nerd-tastic highlights was seeing the actual apparatus Milgram used to examine obedience to authority back in the ’60s.

 

Beyond the obvious awesomeness depicted above, I can point to three key ways I benefited from attending this conference.

1. Networking

I have elaborated on how I went about networking in my previous post. Who I connected with was also critical. At the Teaching Preconference, I met two other research methods textbook authors, including the always-inspiring Beth Morling, and my US-Edition Cozby counterpart Scott Bates. Scott gave an insightful talk on how he is using the APA guidelines for psychology majors to steer his course (re)designs; it was fun to talk with him afterward about course design and how he came to work on the Cozby text too. Turns out both our stories share a similar right place/right time theme. Also, I was delighted to meet a fellow regional representative of the STP’s Early Career Psychologist Council, Ali O’Malley at Butler University.

For the rest of the conference, my networking opportunities were largely driven by Twitter conversations and chance encounters. For example, I happened to sit next to my UBC colleague Eric Eich during the good data practices symposium (see below). He’s the editor in chief of our field’s top journal, Psychological Science. Sitting next to him meant I met Bobbie Spellman, editor in chief of Perspectives On Psychological Science (PoPS). As speaker after speaker pointed to journal editors as the key to improving our field, it was interesting to consider their perspectives too. Tweeting regularly meant that people recognized me, I could learn what was happening in other symposia, and I had conversations in person and online that would not have been possible otherwise. Check out this Twitter feed for what was said by everyone using #aps2013dc.

2. Insight into Good Data Practices, Replicability, and Data Ethics

One of the major reasons I wanted to attend APS this year was the theme program on Building a Better Psychological Science. These speakers built on their contributions to a recent issue of PoPS focusing squarely on this topic.  It was fascinating to hear Daniel Kahneman recall his early days as a psychologist, during which he gave up on a line of research that seemed profitable but he couldn’t replicate to his high standards. Apparently, his own introspection on how he could have been so convinced his small sample studies would replicate became the basis for his later revolutionary and Nobel Prize-winning work with Amos Tversky. Seems to have been a good choice.

Collectively, this lengthy line-up of speakers helped me reconsider the big-picture costs of the way our field has under-valued exact replications in favour of conceptual replications, and how we’ve been chasing p-values below .05. The arguments weren’t really new, but packaging them together in this symposium was impactful for me. Now that I’m a textbook author preparing for my 2nd edition, I found myself constantly considering how we can help our 2nd year psych majors learn best practices. For example, why not include an ethics unit in a statistics course? There’s usually one in research methods… but why not stats too, dealing with data management and reporting and fraud? Also, I’ll be changing the way I discuss the value of exact replications in my next edition. In courses, we could require our students to submit their data and analyses along with their final papers, to be able to reward data archiving and get our students into the habit of submitting raw data to back up claims. So much to (re)consider…

Talking to colleagues about the teaching-related implications of the Good Data Practices symposium was a fascinating exercise. It seems obvious to me that we should be “raising” our majors on best practices, yet there was more variability in reactions to this idea than I had expected. Many colleagues seemed to have simply not considered implications at an undergraduate level, but agreed with me that it’s a good place to start. In a couple of cases, I was challenged about whether such efforts would have any effect. These challenges have compelled me to consider writing a why bother/how-to piece to submit for publication. (Oh summer, could you give me an extra month?)

3. Ideas for my classes

Throughout the entire conference and preconference, I had tons of ideas for my courses! I mentioned a few above regarding methods and stats. In addition, Laura King helped me re-think how I approach Personality Psychology in intro: “you wouldn’t spend a full day discussing phrenology in the biological psychology unit, so why spend a full day on Freud in the personality unit?” Wow. That hit home. I do incorporate modern science of personality in intro, but I will be making major changes to that unit next year.

In her preconference keynote address, Beth Morling offered a ton of specific ideas for intro and methods, centred on her priority of creating effective consumers of psychological research. Specifically, she promotes teaching students to identify the type of claim being made (frequency, association, or causal), and asking four questions about the validity of the study (construct, internal, external, statistical). Which of the four validities is most important depends on the type of claim. See her blog for a bevy of excellent examples. This consumer-oriented approach will inform how I teach methods in intro in particular, where I’m not training psych majors.

Talks by Scott Bates and Keith Stanovich prompted me to strongly consider changing my supplemental readings for research methods. In addition to his course design process, Bates offered a great resource on the purpose of APA style as an indicator of values in the discipline. This article dovetails with conversations a colleague and I have had about this very topic (Jaclyn Rea of our Arts Studies in Research and Writing department). I am strongly considering incorporating this discussion into methods this fall. Usually, I assign parts of Stanovich’s text How to Think about Psychology. I’ve always known it’s unpopular with many students due to its long-windedness and general lack of pedagogical features to help students learn from it. I value his ideas, which was why I’ve been assigning it for years, but the packaging just doesn’t compel my students to read it. I must say that hearing him speak convinced me that although his ideas are insightful he seems to have little sense of audience–a feature reflected in his book, too. Although I’ll still draw from and cite his book, I may be replacing those readings with the APA style article and some other big-picture perspectives on replicability/methods.

Another idea came to me at a talk by Jason Priem, who is promoting his altmetrics approach to measuring scholarly impact. I’d like to create a shareable library for each course in Zotero including all the papers I cite during class. Students can follow up on any article they want, and I can always find what I’ve cited. It’ll be a lot of work to set up, but once it’s there, it’ll be a rich resource that’s easy to update and share.

****

What a fun exercise this was to reflect on my notes and memories from this conference! I highly recommend this process to anyone hoping to actually follow up on rich conference experiences. I am pleasantly surprised by how much I gained from going to APS. I had feared it would be just too big and overwhelming to extract anything meaningful, but that fear was wholly unfounded.

Thanks for reading. Have an experimint.

Ten Simple Steps to Conference Networking

Networking opportunities abounded at the Association for Psychological Science (APS) convention last week. As I was reflecting on the week’s general highlights, it soon became clear that effective networking was the key to most. Let me be clear here: I’ve been going to conferences for about ten years, and I have a long history of being nervous about networking at conferences (still am!). At this point, I suspect it’s some leftover grad school/new faculty imposter syndrome, but it doesn’t really matter why it happens. What matters is I’m not letting my nervousness get in my way too much anymore.

One of the symposia I went to last week specifically addressed professional networking (sponsored by WICS). Perhaps because of this symposium, I have been reflecting on what I think led to my networking successes at APS. In case it’s helpful to someone else, here are ten things I did this week that I think helped me build my connections. Of course, this isn’t meant to be an exhaustive list.

My Ten (Relatively) Simple Steps to Networking

  1. Tweeted regularly. Hands down, best decision. Led to meeting people IRL (read: in real life), and I was recognized by others at times when I didn’t say a word out loud. Importantly, I was re-tweeted, sometimes by APS, Society for the Teaching of Psychology and some major names in the field (thanks APS, STP, Hal Pashler, and Bobbie Spellman!). That means all their followers can see what I wrote. Plus, tweeting helped me stay focused and engaged throughout the conference, and I now have a searchable record of my own (see @cdrawn) and others’ (see #aps2013dc) conference-related thoughts.
  2. Thanked a few speakers and introduced myself, briefly noting a common connection or a sound-bite about why I was interested in their talks. Each time, I met someone else too, because someone else was either standing there who knew the speaker, or because the speaker said “oh, then you should meet…” in response to my sound-bite.
  3. Went to a post-preconference reception. Followed up, elaborated on an earlier brief encounter (see #2). Had ONE glass of wine, which I sipped so slowly it wouldn’t have had a physiological effect, but it relaxed me a bit and helped me switch into a more social mode of conversation.
  4. Grabbed the chance to sit next to a colleague from my institution (who is well-known across the discipline), who sat next to someone else who was well-known across the discipline. Got to know both a bit better through in-session on-topic (and in-joke!) whispers.
  5. Went to a big-picture session everyone seems to be talking about (in this case, replicability). I knew a bit about it beforehand so I knew the key players and ideas. This meant I could tweet responsibly and thoughtfully on this hot topic, and could contribute meaningful whispers (see #4).
  6. Stumbled through asking that first question early in the conference in a small symposium so I’d be more confident and more articulate when asking the next one.
  7. Introduced myself to the speaker/crowd before asking a post-talk question (only after completing #6!). In one case, this meant I gained a couple of followers on twitter, including the speaker, who then tweet-suggested I meet his collaborator on the project who happens to live in my home city.
  8. Hung out in the lobby lounge. A friend wandered past, who then stopped and introduced me to his colleague. Made dinner plans (see #9).
  9. Went to sessions solo. Dined with friends. Years ago I would go to a session because my friend was going to it. I’ve learned to spend conference days largely on my own so I can take advantage of spontaneous conversations as needed, follow my own main interests and odd curiosities, and take quiet breaks when I need them. Dining with friends is then especially fun as we share our most interesting tidbits from the day… or not, if we need a brain break! (This is why it’s important to dine with friends IMO, so no need to worry about impressing them. I can’t spend all day and all evening being “on”.) As Lynn Liben at the WICS symposium noted, networking with peers is important: They start as “siblings” but “grow-up” to become the field.
  10. Followed-up on a twitter exchange. One of the challenges/strengths of writing on Twitter is that it’s limited to 140 characters (10 of which are used by the hashtag required so you’re part of the conversation). Misunderstanding is a real risk. During the conference, I responded to a tweet that I thought had really misrepresented a speaker’s intent, and I hoped it came across politely (after all, I am Canadian!). When I saw the person IRL (see #1), I approached him with a friendly Hi!. He knew immediately who I was, and we were able to further our discussion and finish it amicably.

We can all read other peoples’ papers. In my mind, the real advantage of conferences is the people who are there. Be bold, even just a little, and take advantage of the networking opportunities that come your way. Now, shall we continue this conversation on Twitter? I’m @cdrawn