Tag Archives: research methods

2013/2014 Student Evaluations Response Part 3: Psyc 217

Thank you to each of my students who took the time to complete a student evaluation of teaching this year. I value hearing from each of you, and every year your feedback helps me to become a better teacher. As I explained here, I’m writing reflections on the qualitative and quantitative feedback I received from each of my courses.

 

Of all the courses I teach to learners, Research Methods is my oldest. Over the past 7 years I have taught 12 groups of learners (N = 846)! The core design has largely stayed the same, but I have made many changes on the basis of my own reflections, my adapting knowledge of the topic and our discipline, and—crucially—feedback from students. Quantitative feedback from the student evaluations has remained high this year (see the graph below for a comparison across years).

Psyc217historicUMIs.2008.2014

Now that I have used my own textbook for two years, I was able to explore quantitative ratings on the item “How would you rate the contribution of textbook(s) and assigned readings to this course?” As noted in the graph below, there seems to be a small shift favouring Cozby & Rawn, Canadian Edition (M=4.10 across four sections), over Cozby’s older editions (M=3.84 across 8 sections). Is the textbook perceived a bit more positively because it’s a better book or because I’m both an author and the course instructor? To this point, a few students made comments like this: “I really liked that you were the author of the textbook, it helped connect the course to the reading.” This kind of comment makes me wonder if my colleagues’ data show this shift as well. (I need a control series design rather than just an interrupted time series!)

Psyc 217 contribution of textbook

The CozbyRawn textbook is the core text for this course, but I’ve always supplemented the nuts-and-bolts style book with secondary readings. Up until last year, all secondary readings were from the Stanovich text, but they had repeatedly been received poorly by students (see last year’s reflection). This year, I replaced most Stanovich readings with a few articles highlighting major issues being hotly discussed in our field (e.g., replication). Student feedback about this change was quite positive. Some students noted how the new topics/readings helped them understand current issues in the field, whereas others appreciated fewer readings overall and fewer Stanovich ones. I also experienced the change as a productive and helpful one that improved the course. Thanks to past years’ student feedback for triggering that change!

When I step back and look at the overall set of comments, there are some topics mentioned repeatedly. The one criticism that emerged was the midterms: a number of people found them too long and/or difficult. Because the class averages are in the range required by our department, the exam difficulty seems commensurate with student learning in my course. There were also some requests for a study guide, which is interesting because one does exist. I can’t vouch for the quality of it, but it’s available: http://highered.mcgraw-hill.com/sites/0071056734/student_view0/index.html I’ll make a note to advertise its existence in the syllabus. On the positive side, students found my enthusiasm and approach helpful, including sharing my past mistakes and communicating clear goals/learning objectives. Here’s a comment that sums up some of these themes:

“Who knew research methods could be so interesting? Lectures were consistently energetic and engaging and always promoted critical thinking. I really love that you incorporated contemporary issues in the psychology field into the course content – it has been very useful in interpreting content outside of this class. Continue to do that! Thanks for a great class!”

For next year, I’d like to strive to lecture less and have more in-class activities where people are using the material. I use active learning techniques frequently, but there are some topics that could benefit from revision (quasi-experiments comes to mind). I also intend to revisit the supplementary readings, see if I can replace the few remaining Stanovich chapters with articles available online, and update the “current” readings with ones published since last year (e.g., drawing from an even more recent Perspectives on Psychological Science issue). I’m also working on some bigger things that would span all sections, like adding a big data collection evening, and publishing all the abstracts in some form for next year’s students to be able to refer to.

My Fall 2013 Syllabi

Check out my Fall 2013 syllabi! After spending a lot of time thinking about these courses this summer, I’m excited to share my new syllabi. Psyc 217 features heavily revised Course Goals, new supplemental readings, and a References section listing research that I used to make decisions about this course (e.g., design, policies). Psyc 101 features revised Course Goals and new regular small writing assignments to replace a paper I assigned in years past. I also developed a graphic to help explain the new short writing assignment process.

Psyc 101 Section 005

  • MWF 12-1pm in CIRS 1250
  • 360 students
  • Teaching Assistants: Sara Knauft and Stef Bourrier

Psyc 217 Research Methods Sections 001 and 002

  • MWF 9-10am (Section 1) and 10-11am (Section 2)
  • 92 students per section
  • Teaching Fellows: Allison Brennan, Julia Kam, Jennifer Lay, and Eleni Nasiopolous

See you at Orientations and Imagine on Tuesday, and in class on Wednesday!

Student Evaluations of Teaching 2012/2013: Part 2 Research Methods

Thank you to each of my students who took the time to complete a student evaluation of teaching this year. I value hearing from each of you, and every year your feedback helps me to become a better teacher. As I explained here, I’m writing reflections on the qualitative and quantitative feedback I received from each of my courses.

Psyc 217: Research Methods

I went into this year especially excited about research methods, as I got to use a textbook with my own name on it! Wow, what a thrill! Perhaps reflecting this extra-potent boost of enthusiasm, my quantitative results were overwhelmingly positive this year. Interestingly, I seem to have connected especially effectively with my 10am section. Ratings from my 9am section were positive too (see the mint green bars on the graph linked above)… and on par with years past. But scores from my 10am section were the highest I’ve ever received (see the light purple bars on the graph)! Because I taught the two sections pretty much the same way, I’m not sure what can account for the difference. Suffice it to say, in my mind it was an especially awesome year… and many of my students seem to have felt that way too.

When I teach research methods, it’s often at 9 and 10 in the morning, and I do my very best every day to bring the energy. For many people, this material isn’t exactly inherently exciting. As one student wrote, “Based on what I’ve heard from friends and acquaintances at UBC, research methods is one of the most disliked courses offered at the university due to its sheer boringness.” Thankfully, this student continued, “that said, this instructor did a phenomenal job of teaching the course in a way that students found the material relevant and exciting (to the extent that this material can be exciting).” Such an assessment is the most common comment coming from my student evaluations in this course: Students expect this material to be dull, but I bring it alive. That’s exactly what I strive to do every single day. I’m satisfied that my well-caffeinated efforts are effective for my students.

A few other topics were noted by small subsets of students. Two topics drew ambivalent assessments: groupwork and in-class activities. People seem to have a love-hate relationship with groupwork. First, only a handful of people mentioned it at all, leading me to suspect that mostly people feel neutrally about it (perhaps recognizing its inherent challenges and strengths). The people who noted liking the team project still found it a lot of work, but recognized the value in it. The people who didn’t seem to work as well with teammates report viewing it as a frustrating waste of time. Each year I hear this dichotomous assessment. One thing I tried out last year in response to one particularly struggling group was a mediation meeting, during which I acted as mediator. It seemed to work well to get that group through effectively to the end of the course. To broaden this service and reach the struggling groups I don’t hear about, I am creating a form-based mediation request process for this year. That may sound like a cold approach, but I’ve given it much thought. After years of imploring people to come to me face-to-face to help solve their group challenges, I note that very few groups—or individuals struggling within groups—ever come to me. By formalizing this process, I hope to remove some of the emotion around “tattling,” and treat it as just another issue that needs to be dealt with, just like a grade change request. Hopefully this new process will help reach those extra few groups who are struggling on their own so they can move forward and perform well in group tasks.

In-class activities make material memorable, illustrate difficult concepts, up the energy and attention levels, and make learning fun. Every year, dozens of students report appreciating them. However, there is a small minority of students who don’t appreciate the time spent on these active learning adventures (yes, that’s the subtitle of my blog… see where I’m headed). I’m committed to student learning, and one of the hallmarks of my teaching philosophy is to get out of the way. And data supports my commitment to using active learning techniques (Armbruster, Patel, Johnson, & Weiss, 2009; Deslauriers, Schelew, & Wieman, 2011; Hake, 1998; Prince, 2004; Ruhl, Hughes, & Schloss, 1987; Yoder & Hochevar, 2005). I encourage people who are considering taking research methods or statistics with me (or any of my courses, really), to be ready to engage actively during class. If you’re not up for having fun while learning, my section might not be for you.

The last topic I’ll touch on is the textbooks. A few students noted how much they found my textbook worthwhile (yay!), with one student going so far as to say “I loved her book she wrote, very clear, informative, concise, probably the best textbook I ever used and read due to how clear it is to understand, with all the learning objectives.” I can’t take full credit for that readability (thanks to Cozby for laying such a strong foundation in his 10 prior editions!), yet I’m glad this text is being perceived as helpful. Unfortunately, the Stanovich text once again was voted unhelpful. The messages are useful, but even I find many examples dated and the chapters too lengthy for the points they make. Two years ago I wrote learning objectives and emphasized “get in, find what you need to know, and get out approach” in an attempt to make Stanovich’s text more accessible. Since then, there have been fewer complaints about Stanovich’s text, but a small, consistent group remain. I’ve been back-and-forth on this text for quite a while now, and I’m strongly considering replacing it with a few key peer reviewed articles/commentaries. I have some deep thinking to do in the coming weeks!

Many thanks to all my Psyc 217 students in 2012/2013 students who completed this evaluation. The response rate this year was 67% across both sections, which is my highest rate ever. And thanks to everyone for a really fun year of learning about research methods!

References

Armbruster, P., Patel, M. Johnson, E., & Weiss, M. (2009). Active learning and student-centred pedagogy improve student attitudes and performance in introductory biology. CBE—Life Sciences Education, 8, 203-213.

Deslauriers, L., Schelew, E., & Wieman, C. (2011). Improved learning in a large-enrollment physics class. Science, 332, 862-864.

Hake, R. (1998). Interactive-engagement vs. traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66, 64-74.

Prince, M. (2004). Does active learning work? A review of the research. Journal of Engineering Education, 93, 223-231.

Ruhl, K., Hughes, C., & Schloss, P. (1987). Using the pause procedure to enhance lecture recall. Teacher Education and Special Education, 10, 14-18.

Yoder, J. D., & Hochevar, C. M. (2005). Encouraging active learning can improve students’ performance on examinations. Teaching of Psychology, 32, 91-95.

Reflecting on APS

Last week I attended the Association for Psychological Science Convention for the first time, including the Society for the Teaching of Psychology  (STP) Teaching Preconference. I am grateful to STP for awarding me an Early Career Travel Grant to help me offset the cost of my flight all the way from Vancouver Canada to Washington DC.

One of my nerd-tastic highlights was seeing the actual apparatus Milgram used to examine obedience to authority back in the ’60s.

 

Beyond the obvious awesomeness depicted above, I can point to three key ways I benefited from attending this conference.

1. Networking

I have elaborated on how I went about networking in my previous post. Who I connected with was also critical. At the Teaching Preconference, I met two other research methods textbook authors, including the always-inspiring Beth Morling, and my US-Edition Cozby counterpart Scott Bates. Scott gave an insightful talk on how he is using the APA guidelines for psychology majors to steer his course (re)designs; it was fun to talk with him afterward about course design and how he came to work on the Cozby text too. Turns out both our stories share a similar right place/right time theme. Also, I was delighted to meet a fellow regional representative of the STP’s Early Career Psychologist Council, Ali O’Malley at Butler University.

For the rest of the conference, my networking opportunities were largely driven by Twitter conversations and chance encounters. For example, I happened to sit next to my UBC colleague Eric Eich during the good data practices symposium (see below). He’s the editor in chief of our field’s top journal, Psychological Science. Sitting next to him meant I met Bobbie Spellman, editor in chief of Perspectives On Psychological Science (PoPS). As speaker after speaker pointed to journal editors as the key to improving our field, it was interesting to consider their perspectives too. Tweeting regularly meant that people recognized me, I could learn what was happening in other symposia, and I had conversations in person and online that would not have been possible otherwise. Check out this Twitter feed for what was said by everyone using #aps2013dc.

2. Insight into Good Data Practices, Replicability, and Data Ethics

One of the major reasons I wanted to attend APS this year was the theme program on Building a Better Psychological Science. These speakers built on their contributions to a recent issue of PoPS focusing squarely on this topic.  It was fascinating to hear Daniel Kahneman recall his early days as a psychologist, during which he gave up on a line of research that seemed profitable but he couldn’t replicate to his high standards. Apparently, his own introspection on how he could have been so convinced his small sample studies would replicate became the basis for his later revolutionary and Nobel Prize-winning work with Amos Tversky. Seems to have been a good choice.

Collectively, this lengthy line-up of speakers helped me reconsider the big-picture costs of the way our field has under-valued exact replications in favour of conceptual replications, and how we’ve been chasing p-values below .05. The arguments weren’t really new, but packaging them together in this symposium was impactful for me. Now that I’m a textbook author preparing for my 2nd edition, I found myself constantly considering how we can help our 2nd year psych majors learn best practices. For example, why not include an ethics unit in a statistics course? There’s usually one in research methods… but why not stats too, dealing with data management and reporting and fraud? Also, I’ll be changing the way I discuss the value of exact replications in my next edition. In courses, we could require our students to submit their data and analyses along with their final papers, to be able to reward data archiving and get our students into the habit of submitting raw data to back up claims. So much to (re)consider…

Talking to colleagues about the teaching-related implications of the Good Data Practices symposium was a fascinating exercise. It seems obvious to me that we should be “raising” our majors on best practices, yet there was more variability in reactions to this idea than I had expected. Many colleagues seemed to have simply not considered implications at an undergraduate level, but agreed with me that it’s a good place to start. In a couple of cases, I was challenged about whether such efforts would have any effect. These challenges have compelled me to consider writing a why bother/how-to piece to submit for publication. (Oh summer, could you give me an extra month?)

3. Ideas for my classes

Throughout the entire conference and preconference, I had tons of ideas for my courses! I mentioned a few above regarding methods and stats. In addition, Laura King helped me re-think how I approach Personality Psychology in intro: “you wouldn’t spend a full day discussing phrenology in the biological psychology unit, so why spend a full day on Freud in the personality unit?” Wow. That hit home. I do incorporate modern science of personality in intro, but I will be making major changes to that unit next year.

In her preconference keynote address, Beth Morling offered a ton of specific ideas for intro and methods, centred on her priority of creating effective consumers of psychological research. Specifically, she promotes teaching students to identify the type of claim being made (frequency, association, or causal), and asking four questions about the validity of the study (construct, internal, external, statistical). Which of the four validities is most important depends on the type of claim. See her blog for a bevy of excellent examples. This consumer-oriented approach will inform how I teach methods in intro in particular, where I’m not training psych majors.

Talks by Scott Bates and Keith Stanovich prompted me to strongly consider changing my supplemental readings for research methods. In addition to his course design process, Bates offered a great resource on the purpose of APA style as an indicator of values in the discipline. This article dovetails with conversations a colleague and I have had about this very topic (Jaclyn Rea of our Arts Studies in Research and Writing department). I am strongly considering incorporating this discussion into methods this fall. Usually, I assign parts of Stanovich’s text How to Think about Psychology. I’ve always known it’s unpopular with many students due to its long-windedness and general lack of pedagogical features to help students learn from it. I value his ideas, which was why I’ve been assigning it for years, but the packaging just doesn’t compel my students to read it. I must say that hearing him speak convinced me that although his ideas are insightful he seems to have little sense of audience–a feature reflected in his book, too. Although I’ll still draw from and cite his book, I may be replacing those readings with the APA style article and some other big-picture perspectives on replicability/methods.

Another idea came to me at a talk by Jason Priem, who is promoting his altmetrics approach to measuring scholarly impact. I’d like to create a shareable library for each course in Zotero including all the papers I cite during class. Students can follow up on any article they want, and I can always find what I’ve cited. It’ll be a lot of work to set up, but once it’s there, it’ll be a rich resource that’s easy to update and share.

****

What a fun exercise this was to reflect on my notes and memories from this conference! I highly recommend this process to anyone hoping to actually follow up on rich conference experiences. I am pleasantly surprised by how much I gained from going to APS. I had feared it would be just too big and overwhelming to extract anything meaningful, but that fear was wholly unfounded.

Thanks for reading. Have an experimint.

Welcome back!

September 2012 is here, and the first week is already over! I forgot how tired I feel by Friday afternoons — wow! It’s like I can feel my body powering down. But my fatigue is warranted. This has been such a fun week! My husband and I kicked off the school year with a whirlwind trip to Ontario to witness my friend’s wedding last weekend. She and I lived together all through undergrad, and were basically inseparable during that time. A few of our other lovely friends from undergrad were there too, so it was wonderful to catch up and reminisce about good old Waterloo. The experience also served to remind me of what a profound impact my undergraduate experience had on my life, both intellectually and socially… which got me pumped to be a part of other people’s undergraduate experience!

On Tuesday I arrived in class at 5pm… and sat down in the student chairs. Yes, I’m taking a course! Why, you ask? Well, I want to. I value the classroom as a rich opportunity to learn, and I felt it was time to sit down and feel what it’s like to learn in that way again. I chose the course Psyc 312A: History of Psychology for many reasons. First, Dr. Andrea Perrino is amazing. I was a TA for her a long time ago and was inspired then by her enthusiasm. I wanted the chance to learn from her, as one teacher to another, to consider her pedagogical choices and prompt me to reflect on my own. Faculty rarely ever watch each other teach; I was grateful she agreed I could take her class for this rich opportunity. Second, I am (finally!) interested in the content! I’ve been teaching intro psych (3 years and counting), research methods (5 years and counting), and statistics (recently renewed after a few years’ hiatus). These are broad, generalist courses: my training in my home area of social psychology is useful but not always directly. Over these years these courses have prompted me to cultivate an interest in the discipline as a whole, and I felt it was time for me to really consider the origins and development of my discipline to enrich how and what I teach in these generalist courses. Psychology is only about 150 years old, so it shouldn’t be too hard, right? Ha!

One of the things Andrea (I mean… Dr. Perrino) did on her first day was started foremost with an introduction the the history of psychology. The topic. Not the syllabus. Sure, that came later, but she kicked off with a passionate rationale for why this course is important. It was inspiring, and influenced the way I began my courses on Wednesday. Instead of starting with the syllabus, I started with the topic, the reasons why it’s important to take this course. Based on feedback after that first class, I seem to have succeeded in inspiring at least some students to be excited about our course (even research methods!). In fact, I barely covered the syllabus at all in Intro, but did so using an i>clicker quiz today. I did that last year too, but this year I was more deliberate in my choice of what to reveal on the first day versus the second. The subject is most important… how we get there is important too, but secondary.

My students have been fantastic this week! Research methods felt like a class reunion from last year’s intro — fantastic to see so many of my fabulous students returning for more psychology, and I can’t wait to meet everyone else! And intro… well… I’ve never had two completely-filled hours of student meetings on the first day of class! I have had such fun meeting so many students this week. Their energy is palpable: it’s a new year, a new beginning, and we’re going to have a great time!

Here’s (sort of) what I see when I look out from the front of my intro psych classroom: 270ish energized students! Click on the image to enlarge it. See you next week!