Tag Archives: textbook

2013/2014 Student Evaluations Response Part 3: Psyc 217

Thank you to each of my students who took the time to complete a student evaluation of teaching this year. I value hearing from each of you, and every year your feedback helps me to become a better teacher. As I explained here, I’m writing reflections on the qualitative and quantitative feedback I received from each of my courses.

 

Of all the courses I teach to learners, Research Methods is my oldest. Over the past 7 years I have taught 12 groups of learners (N = 846)! The core design has largely stayed the same, but I have made many changes on the basis of my own reflections, my adapting knowledge of the topic and our discipline, and—crucially—feedback from students. Quantitative feedback from the student evaluations has remained high this year (see the graph below for a comparison across years).

Psyc217historicUMIs.2008.2014

Now that I have used my own textbook for two years, I was able to explore quantitative ratings on the item “How would you rate the contribution of textbook(s) and assigned readings to this course?” As noted in the graph below, there seems to be a small shift favouring Cozby & Rawn, Canadian Edition (M=4.10 across four sections), over Cozby’s older editions (M=3.84 across 8 sections). Is the textbook perceived a bit more positively because it’s a better book or because I’m both an author and the course instructor? To this point, a few students made comments like this: “I really liked that you were the author of the textbook, it helped connect the course to the reading.” This kind of comment makes me wonder if my colleagues’ data show this shift as well. (I need a control series design rather than just an interrupted time series!)

Psyc 217 contribution of textbook

The CozbyRawn textbook is the core text for this course, but I’ve always supplemented the nuts-and-bolts style book with secondary readings. Up until last year, all secondary readings were from the Stanovich text, but they had repeatedly been received poorly by students (see last year’s reflection). This year, I replaced most Stanovich readings with a few articles highlighting major issues being hotly discussed in our field (e.g., replication). Student feedback about this change was quite positive. Some students noted how the new topics/readings helped them understand current issues in the field, whereas others appreciated fewer readings overall and fewer Stanovich ones. I also experienced the change as a productive and helpful one that improved the course. Thanks to past years’ student feedback for triggering that change!

When I step back and look at the overall set of comments, there are some topics mentioned repeatedly. The one criticism that emerged was the midterms: a number of people found them too long and/or difficult. Because the class averages are in the range required by our department, the exam difficulty seems commensurate with student learning in my course. There were also some requests for a study guide, which is interesting because one does exist. I can’t vouch for the quality of it, but it’s available: http://highered.mcgraw-hill.com/sites/0071056734/student_view0/index.html I’ll make a note to advertise its existence in the syllabus. On the positive side, students found my enthusiasm and approach helpful, including sharing my past mistakes and communicating clear goals/learning objectives. Here’s a comment that sums up some of these themes:

“Who knew research methods could be so interesting? Lectures were consistently energetic and engaging and always promoted critical thinking. I really love that you incorporated contemporary issues in the psychology field into the course content – it has been very useful in interpreting content outside of this class. Continue to do that! Thanks for a great class!”

For next year, I’d like to strive to lecture less and have more in-class activities where people are using the material. I use active learning techniques frequently, but there are some topics that could benefit from revision (quasi-experiments comes to mind). I also intend to revisit the supplementary readings, see if I can replace the few remaining Stanovich chapters with articles available online, and update the “current” readings with ones published since last year (e.g., drawing from an even more recent Perspectives on Psychological Science issue). I’m also working on some bigger things that would span all sections, like adding a big data collection evening, and publishing all the abstracts in some form for next year’s students to be able to refer to.

Student Evaluations of Teaching 2012/2013: Part 2 Research Methods

Thank you to each of my students who took the time to complete a student evaluation of teaching this year. I value hearing from each of you, and every year your feedback helps me to become a better teacher. As I explained here, I’m writing reflections on the qualitative and quantitative feedback I received from each of my courses.

Psyc 217: Research Methods

I went into this year especially excited about research methods, as I got to use a textbook with my own name on it! Wow, what a thrill! Perhaps reflecting this extra-potent boost of enthusiasm, my quantitative results were overwhelmingly positive this year. Interestingly, I seem to have connected especially effectively with my 10am section. Ratings from my 9am section were positive too (see the mint green bars on the graph linked above)… and on par with years past. But scores from my 10am section were the highest I’ve ever received (see the light purple bars on the graph)! Because I taught the two sections pretty much the same way, I’m not sure what can account for the difference. Suffice it to say, in my mind it was an especially awesome year… and many of my students seem to have felt that way too.

When I teach research methods, it’s often at 9 and 10 in the morning, and I do my very best every day to bring the energy. For many people, this material isn’t exactly inherently exciting. As one student wrote, “Based on what I’ve heard from friends and acquaintances at UBC, research methods is one of the most disliked courses offered at the university due to its sheer boringness.” Thankfully, this student continued, “that said, this instructor did a phenomenal job of teaching the course in a way that students found the material relevant and exciting (to the extent that this material can be exciting).” Such an assessment is the most common comment coming from my student evaluations in this course: Students expect this material to be dull, but I bring it alive. That’s exactly what I strive to do every single day. I’m satisfied that my well-caffeinated efforts are effective for my students.

A few other topics were noted by small subsets of students. Two topics drew ambivalent assessments: groupwork and in-class activities. People seem to have a love-hate relationship with groupwork. First, only a handful of people mentioned it at all, leading me to suspect that mostly people feel neutrally about it (perhaps recognizing its inherent challenges and strengths). The people who noted liking the team project still found it a lot of work, but recognized the value in it. The people who didn’t seem to work as well with teammates report viewing it as a frustrating waste of time. Each year I hear this dichotomous assessment. One thing I tried out last year in response to one particularly struggling group was a mediation meeting, during which I acted as mediator. It seemed to work well to get that group through effectively to the end of the course. To broaden this service and reach the struggling groups I don’t hear about, I am creating a form-based mediation request process for this year. That may sound like a cold approach, but I’ve given it much thought. After years of imploring people to come to me face-to-face to help solve their group challenges, I note that very few groups—or individuals struggling within groups—ever come to me. By formalizing this process, I hope to remove some of the emotion around “tattling,” and treat it as just another issue that needs to be dealt with, just like a grade change request. Hopefully this new process will help reach those extra few groups who are struggling on their own so they can move forward and perform well in group tasks.

In-class activities make material memorable, illustrate difficult concepts, up the energy and attention levels, and make learning fun. Every year, dozens of students report appreciating them. However, there is a small minority of students who don’t appreciate the time spent on these active learning adventures (yes, that’s the subtitle of my blog… see where I’m headed). I’m committed to student learning, and one of the hallmarks of my teaching philosophy is to get out of the way. And data supports my commitment to using active learning techniques (Armbruster, Patel, Johnson, & Weiss, 2009; Deslauriers, Schelew, & Wieman, 2011; Hake, 1998; Prince, 2004; Ruhl, Hughes, & Schloss, 1987; Yoder & Hochevar, 2005). I encourage people who are considering taking research methods or statistics with me (or any of my courses, really), to be ready to engage actively during class. If you’re not up for having fun while learning, my section might not be for you.

The last topic I’ll touch on is the textbooks. A few students noted how much they found my textbook worthwhile (yay!), with one student going so far as to say “I loved her book she wrote, very clear, informative, concise, probably the best textbook I ever used and read due to how clear it is to understand, with all the learning objectives.” I can’t take full credit for that readability (thanks to Cozby for laying such a strong foundation in his 10 prior editions!), yet I’m glad this text is being perceived as helpful. Unfortunately, the Stanovich text once again was voted unhelpful. The messages are useful, but even I find many examples dated and the chapters too lengthy for the points they make. Two years ago I wrote learning objectives and emphasized “get in, find what you need to know, and get out approach” in an attempt to make Stanovich’s text more accessible. Since then, there have been fewer complaints about Stanovich’s text, but a small, consistent group remain. I’ve been back-and-forth on this text for quite a while now, and I’m strongly considering replacing it with a few key peer reviewed articles/commentaries. I have some deep thinking to do in the coming weeks!

Many thanks to all my Psyc 217 students in 2012/2013 students who completed this evaluation. The response rate this year was 67% across both sections, which is my highest rate ever. And thanks to everyone for a really fun year of learning about research methods!

References

Armbruster, P., Patel, M. Johnson, E., & Weiss, M. (2009). Active learning and student-centred pedagogy improve student attitudes and performance in introductory biology. CBE—Life Sciences Education, 8, 203-213.

Deslauriers, L., Schelew, E., & Wieman, C. (2011). Improved learning in a large-enrollment physics class. Science, 332, 862-864.

Hake, R. (1998). Interactive-engagement vs. traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66, 64-74.

Prince, M. (2004). Does active learning work? A review of the research. Journal of Engineering Education, 93, 223-231.

Ruhl, K., Hughes, C., & Schloss, P. (1987). Using the pause procedure to enhance lecture recall. Teacher Education and Special Education, 10, 14-18.

Yoder, J. D., & Hochevar, C. M. (2005). Encouraging active learning can improve students’ performance on examinations. Teaching of Psychology, 32, 91-95.

Reflecting on APS

Last week I attended the Association for Psychological Science Convention for the first time, including the Society for the Teaching of Psychology  (STP) Teaching Preconference. I am grateful to STP for awarding me an Early Career Travel Grant to help me offset the cost of my flight all the way from Vancouver Canada to Washington DC.

One of my nerd-tastic highlights was seeing the actual apparatus Milgram used to examine obedience to authority back in the ’60s.

 

Beyond the obvious awesomeness depicted above, I can point to three key ways I benefited from attending this conference.

1. Networking

I have elaborated on how I went about networking in my previous post. Who I connected with was also critical. At the Teaching Preconference, I met two other research methods textbook authors, including the always-inspiring Beth Morling, and my US-Edition Cozby counterpart Scott Bates. Scott gave an insightful talk on how he is using the APA guidelines for psychology majors to steer his course (re)designs; it was fun to talk with him afterward about course design and how he came to work on the Cozby text too. Turns out both our stories share a similar right place/right time theme. Also, I was delighted to meet a fellow regional representative of the STP’s Early Career Psychologist Council, Ali O’Malley at Butler University.

For the rest of the conference, my networking opportunities were largely driven by Twitter conversations and chance encounters. For example, I happened to sit next to my UBC colleague Eric Eich during the good data practices symposium (see below). He’s the editor in chief of our field’s top journal, Psychological Science. Sitting next to him meant I met Bobbie Spellman, editor in chief of Perspectives On Psychological Science (PoPS). As speaker after speaker pointed to journal editors as the key to improving our field, it was interesting to consider their perspectives too. Tweeting regularly meant that people recognized me, I could learn what was happening in other symposia, and I had conversations in person and online that would not have been possible otherwise. Check out this Twitter feed for what was said by everyone using #aps2013dc.

2. Insight into Good Data Practices, Replicability, and Data Ethics

One of the major reasons I wanted to attend APS this year was the theme program on Building a Better Psychological Science. These speakers built on their contributions to a recent issue of PoPS focusing squarely on this topic.  It was fascinating to hear Daniel Kahneman recall his early days as a psychologist, during which he gave up on a line of research that seemed profitable but he couldn’t replicate to his high standards. Apparently, his own introspection on how he could have been so convinced his small sample studies would replicate became the basis for his later revolutionary and Nobel Prize-winning work with Amos Tversky. Seems to have been a good choice.

Collectively, this lengthy line-up of speakers helped me reconsider the big-picture costs of the way our field has under-valued exact replications in favour of conceptual replications, and how we’ve been chasing p-values below .05. The arguments weren’t really new, but packaging them together in this symposium was impactful for me. Now that I’m a textbook author preparing for my 2nd edition, I found myself constantly considering how we can help our 2nd year psych majors learn best practices. For example, why not include an ethics unit in a statistics course? There’s usually one in research methods… but why not stats too, dealing with data management and reporting and fraud? Also, I’ll be changing the way I discuss the value of exact replications in my next edition. In courses, we could require our students to submit their data and analyses along with their final papers, to be able to reward data archiving and get our students into the habit of submitting raw data to back up claims. So much to (re)consider…

Talking to colleagues about the teaching-related implications of the Good Data Practices symposium was a fascinating exercise. It seems obvious to me that we should be “raising” our majors on best practices, yet there was more variability in reactions to this idea than I had expected. Many colleagues seemed to have simply not considered implications at an undergraduate level, but agreed with me that it’s a good place to start. In a couple of cases, I was challenged about whether such efforts would have any effect. These challenges have compelled me to consider writing a why bother/how-to piece to submit for publication. (Oh summer, could you give me an extra month?)

3. Ideas for my classes

Throughout the entire conference and preconference, I had tons of ideas for my courses! I mentioned a few above regarding methods and stats. In addition, Laura King helped me re-think how I approach Personality Psychology in intro: “you wouldn’t spend a full day discussing phrenology in the biological psychology unit, so why spend a full day on Freud in the personality unit?” Wow. That hit home. I do incorporate modern science of personality in intro, but I will be making major changes to that unit next year.

In her preconference keynote address, Beth Morling offered a ton of specific ideas for intro and methods, centred on her priority of creating effective consumers of psychological research. Specifically, she promotes teaching students to identify the type of claim being made (frequency, association, or causal), and asking four questions about the validity of the study (construct, internal, external, statistical). Which of the four validities is most important depends on the type of claim. See her blog for a bevy of excellent examples. This consumer-oriented approach will inform how I teach methods in intro in particular, where I’m not training psych majors.

Talks by Scott Bates and Keith Stanovich prompted me to strongly consider changing my supplemental readings for research methods. In addition to his course design process, Bates offered a great resource on the purpose of APA style as an indicator of values in the discipline. This article dovetails with conversations a colleague and I have had about this very topic (Jaclyn Rea of our Arts Studies in Research and Writing department). I am strongly considering incorporating this discussion into methods this fall. Usually, I assign parts of Stanovich’s text How to Think about Psychology. I’ve always known it’s unpopular with many students due to its long-windedness and general lack of pedagogical features to help students learn from it. I value his ideas, which was why I’ve been assigning it for years, but the packaging just doesn’t compel my students to read it. I must say that hearing him speak convinced me that although his ideas are insightful he seems to have little sense of audience–a feature reflected in his book, too. Although I’ll still draw from and cite his book, I may be replacing those readings with the APA style article and some other big-picture perspectives on replicability/methods.

Another idea came to me at a talk by Jason Priem, who is promoting his altmetrics approach to measuring scholarly impact. I’d like to create a shareable library for each course in Zotero including all the papers I cite during class. Students can follow up on any article they want, and I can always find what I’ve cited. It’ll be a lot of work to set up, but once it’s there, it’ll be a rich resource that’s easy to update and share.

****

What a fun exercise this was to reflect on my notes and memories from this conference! I highly recommend this process to anyone hoping to actually follow up on rich conference experiences. I am pleasantly surprised by how much I gained from going to APS. I had feared it would be just too big and overwhelming to extract anything meaningful, but that fear was wholly unfounded.

Thanks for reading. Have an experimint.

Ethics in Teaching

This morning I gave a guest lesson in an “Ethics and Professional Issues” seminar for clinical graduate students. Although I encounter ethical issues in teaching routinely, preparing for this guest lesson gave me a chance to deliberately examine all the activities involved in teaching using the lens of ethics. My thinking was greatly informed by the edited volume called Teaching Ethically that came out last year, which also prompted me to examine the APA code of ethics.

I came up with a list of domains in teaching where ethical issues pop up. You might be surprised by some of these, but not by others. Do you have anything to add?

  • Competency
    • Content knowledge (what to teach)
    • Pedagogical knowledge (how best to teach it; evidence-based assessment)
    • Adequate preparation (class, course)
    • Classroom management (e.g., strategies for dealing with sensitive issues like mental illness, gender differences, ugly history of IQ scores)
    • Self-assess “boundaries of competence” and don’t work beyond them (if must, obtain training)
    • Professional development
    • Seeking advice from and collaborating with colleagues to improve learning
    • Continual improvement (self, course, program, degree)
    • Scholarly teaching/SoTL: using new method without adequate research or preparation
  • Fairness
    • Diversity and inclusivity (access to materials including cost of the textbook, self-disclosure in class activities)
    • Assigning a textbook you don’t use (much)
    • Discourage and pursue cases of plagiarism, cheating
    • Textbook cost
    • Grading and evaluation is clear, competently done, consistent
    • Doing the SoTL work including withholding treatment to one group
    • Accommodations for special circumstances: extra grade chances, re-grades, make up a missed exam
    • Accurately describe your course, set up appropriate expectations (e.g., grades)
  • Faculty-student relationships
    • Trust and power
    • Authorship with graduate and undergraduate student collaborators
    • Writing reference letters for students you know have slim chances of getting in somewhere or who you don’t know that well (and not writing letters when you can)
    • Avoiding multiple (conflicting) roles: research supervisor, employer, teacher, mentor, evaluator, researcher (SoTL), TA/teaching supervisor
    • Consider the impact of challenging students’ core beliefs (e.g., God, evolution, trust in authority, gay rights…)
    • Social media: Facebook friends, Twitter following à blurring edge of professional relationship
    • Assigning a textbook when you’re and author and will receive royalties
    • Accepting gifts from students (and textbook publishers, for that matter)
  • Confidentiality
    • Grades files shared over email (security)
    • Sending grades over email to students using non-UBC email addresses
    • Storage and confidential shredding of paper material
    • Non-disclosure to parents
  • Law
    • Understanding copyright laws (including digital copyright)
    • Complying with copyright laws

Psyc 217 Research Methods: What textbooks do you need?

Hello to all my new, eager students! I’ve received quite a few emails recently about the textbooks. Normally these are the kinds of questions that would receive an answer along the lines of  “please check your syllabus” — however, considering I haven’t quite finished it yet, I can’t exactly expect you to consult it.

Here’s what you need:

  • Cozby, P. C., & Rawn, C. D. (2012). Methods in Behavioural Research(Canadian Ed.). Toronto, ON: McGraw-Hill Ryerson.
    • This is a nuts-and-bolts style guide to research methods that focuses on the details of how to conduct research. Available new from the bookstore, or electronically on Coursesmart. *Note that used editions do not exist because this edition is brand new.
    • Yes, I am the second author. Please note that I am donating all royalties from UBC sales to UBC scholarships.
    • Can you use an old edition? No, I do not recommend it. Here are a few reasons why. First off: most examples are changed, updated, and now integrate Canadian culture, terminology, and research (spot your profs in the reference list!). Second, I totally overhauled the ethics chapter to reflect the Canadian context of conducting research (e.g., in terms of government, terminology, structure). The old one is all-American. Third, I’ve improved the book based on two rounds of (Canadian) reviews as well as my own experiences teaching this course for the past four years. You’ll notice a synergy between what happens in class (e.g., diagrams, ways of explaining things), and the textbook. Fourth, I’ve added extra features to help you learn. For example, I’ve re-worked the learning objectives so that it’s clearer what to do with them, and I’ve ensured every bolded term is in the glossary, which wasn’t true before…. Changes like that that will make it easier for you to learn from this text.
  • Stanovich, K. E. (2009). How to Think Straight about Psychology(9th Ed.). Boston, MA: Pearson.
    • This guide to research methods provides a nice complement to the details of the first text. It is written from a bigger picture perspective. Available new and used from the bookstore. If you buy it new from the bookstore, it comes with a $10 off i>clicker coupon and a free guide to APA style.
  • Cuttler, C. (2010). Research Methods in Psychology. Dubuque, IA: Kendall Hunt.
    • This is the lab guide — created for our Psych 217 labs — that will help you and your teammates work step-by-step to create a successful research project.
  • i>clicker Student Response System, available new and used from the bookstore.

Hope that’s a helpful start. I’ll post the syllabus later this week when I have it complete. Looking forward to meeting you next week!