Tag Archives: learning assessments

2013/2014 Student Evaluations Response Part 1/4: Intro Psych

Thank you to each of my students who took the time to complete a student evaluation of teaching this year. I value hearing from each of you, and every year your feedback helps me to become a better teacher. As I explained here, I’m writing reflections on the qualitative and quantitative feedback I received from each of my courses.

 

After teaching students intro psych as a 6-credit full-year course for the past three years, in 2013/2014 I was required to transform it into 101 and 102. Broadly speaking, the Term1/Term2 division from the 6-credit course stays the same, but turning Term 2 of Psyc 100 into a semi-standalone Psyc 102 proved more challenging than converting Term 1 into 101. Because these two courses really still form one unit in my mind, and I structure the courses extremely similarly, I will discuss them in tandem.

Across both courses, quantitative feedback was similar (albeit a bit higher in 101 than 102). Students rated the textbook equally high (4.2 & 4.3/5), which makes sense because I use the same text for both, and many students have told me informally that they enjoy reading the book (some qualify this endorsement with “for a textbook”). Check out my overall UMI scores from this year and all previous years here (click to enlarge, and click here for further discussion of UMIs):

IntroPsycHistoricUMIs.2009.2014

 

In the qualitative feedback, many of the same positive and helpful features were highlighted by students in both courses. Overall, students report enjoying and finding valuable the clickers, classroom discussions (often tied to clicker questions), films, opportunities to apply what is being learned, the 3-midterm format that helps stay on top of things even if it’s slightly annoying to be so frequently tested, music before class, the organization of class periods, my enthusiasm and energy, my effort to learn many students’ names, and the Invitational Office Hours. Capturing many of these commonly-mentioned features, one student from Psyc 102 wrote,

It was incredibly impressive how she tried to learn the name of every single student that she interacted with, despite the size of the class. The IOH were also a surprisingly fun experience. The class was very interactive, which definitely helped me learn, and even though I was unhappy about having three midterms at first I think I have to conclude it made studying for each one much easier and less stressful.”

One new element I added to both courses this year were five mini-papers which I called “Writing to Learn” (W2L) assignments (replacing a single 500 word paper I used to assign to be marked by TAs; see last year’s reflection for rationale). Students picked a topic from each of the two chapters about to be tested, wrote a paragraph explaining and applying the concept to their lives, then read 5 peers’ papers and gave feedback to them using peerScholar software. Students received feedback from their peers, and were able to choose any two topics to improve and reproduce on the final exam (no notes!). Overall, students reported finding the Writing to Learn assignments helpful for learning, and some mentioned that reading others’ work was helpful as well (both of these results are consistent with past research on similar writing assignments and peer review). My TAs have reported being able to grasp whether students knew what they were talking about from the writing section on the exams – and my test scores were higher than in previous years, so the goal of increasing learning seems to have been met! However, of the students who mentioned the W2L assignments, many noted that quality of peer feedback received was low. Dr. Peter Graf and I are just starting a project to deal with this very issue of enhancing peer feedback. It may take a couple of years to figure it out in a way that’s scalable to 300-400 students at a time, but we’re working on it.

Interestingly, a couple of students in each course noted my responses to student incivility. In one case it was failing to follow instructions to complete the bubbles during the exam time given, and another case (mentioned a few times in 102, actually), was my response to students talking in class. Side chatter is really only a problem in my first year courses – and it’s a consistent one that varies in severity year-to-year with different cohorts. Interestingly, Gillian Sandstrom and I have a paper about to come out in Teaching of Psychology showing that some chatter is a good thing: it can build a sense of community in a large class. But it can feel disrespectful and distracting to me. Perhaps I should consider building in even more opportunities for structured conversation, because clearly it’s going to happen anyway.

In 102 this year, rather than devoting a whole week to Chapter 2 Research Methods – which I do in 101 in the same place I did back when I taught the 6-credit version – I decided to split it up and cover topics as they came up throughout the term. For example, I used intelligence testing as a chance to discuss measurement and survey designs, and social psychology (specifically Milgram’s studies) as a chance to address the ethics of deception in research designs. Although I think this was a solid solution in theory, in practice there were definitely times when I felt like I was awkwardly wedging topics in to 102. Indeed, a few students mentioned this flow problem too – and it seems to be students who took both 101 and 102 with me who noticed the difference. Hopefully next year I’ll be able to smooth topics out a bit more effectively, perhaps cutting even more material to make more room for these new topics and ensuing discussions.

Although I still would prefer to teach intro psych as a unified whole with the same students over the whole year, apparently that’s not an option any longer. I have begun the process of converting this course into two halves effectively, and given the feedback above, I think I’m heading in that direction.

Creative Advertisement Showcase 2014

A couple of weeks ago in my section of Psyc 208 we held the Creative Advertisement Showcase, which was a fantastic celebration of what my students had discovered throughout the term! Previously, each team of students identified a learning challenge they face, investigated primary sources for insight into understanding and addressing the challenge, and summarized those sources in annotated bibliographies and team abstracts. (See the Team Project Guide for a full description of this multi-part assignment.) The purpose of the Creative Advertisement was to get the word out to fellow students about research-based techniques and strategies for addressing the learning challenges they face. They truly were creative! Projects ranged from video and live games to posters to live skits and demonstrations to videos and pamphlets… an impressive variety! Check out some of their videos and websites (ordered by team #)…

Team 4’s Sleep Fairy:  https://www.youtube.com/watch?v=PGOWxnPizOk 

Team 5’s Culture Shock: https://www.youtube.com/watch?v=GyZcynf2jk4&feature=youtu.be

Team 22’s Motivation Makeover: https://www.youtube.com/watch?v=l6syynl18XA

Team 11’s Loneliness: https://drive.google.com/folderview?id=0B_qDfS_x4VtvLWhIZXRNS2pxVHc&usp=sharing

Team 18’s Sleep: https://docs.google.com/file/d/0B0xsEQGqsy3IVGczVVpZTGZkcGM/edit

Here’s a shot of Team 19’s Effects of Internet Use on Learning station…

… and one of Team 3’s Social Loafing game based on Apples to Apples

Thanks to everyone for a solid effort on these projects and throughout the term! Study smartly for your finals… and remember that no matter how you do on them, that’s not a reflection of your worth as a person.

Student Evaluations of Teaching 2012/2013: Part 3 Special Topics Psyc 208

Thank you to each of my students who took the time to complete a student evaluation of teaching this year. I value hearing from each of you, and every year your feedback helps me to become a better teacher. As I explained here, I’m writing reflections on the qualitative and quantitative feedback I received from each of my courses.

Psych 208: Special Topics

Overall, I thought this course was pretty smooth. Over the past four years I’ve developed a clearer vision for what this course is (see the syllabus), and I think that’s reflected in greater coherence, greater integration across in-class teaching methods, content, and learning objectives. Learning appraisals are in decent shape (although could probably use some refreshment in the coming year). As I have matured, I have matured this course.

After such self-assessment, I was pleased to note that students rated this course more positively than any previous iteration, right on par with my other more traditional courses (check out the quantitative data here). Reading the qualitative evaluations was almost overwhelming. Student after student noted how useful this course was, how much they applied these concepts to their everyday life, how they built skills they’d take with them into future courses and their careers. I am absolutely blown away by what people said about this course. When I first envisioned this course, I wanted it to be useful. So much of psychology (particularly social psychology) can improve our lives, and that’s exactly what I dreamed this course would help people do: apply our amazing research to improve their lives. To that end, numerous students wrote things like,

I found myself always referencing the course subject matter to my friends and applying it within my own life”, “this has been one of the most beneficial classes of my university career,” and “this will be one of those courses that I look back on knowing that it was a good use of my time” – even if they noted they were about to graduate.

All of this positivity was despite (or because of?) the extent to which this course challenged students in various ways (e.g., “not an easy A”) yet was perceived as valuable (e.g., “the course project on group work is such a valuable skill that students need”).  The usefulness of our course material got through to these students, and I’m absolutely thrilled!

Given these positive comments, I think it’s worthwhile to note that as I’ve matured (with) this course, there’s one key tweak I made in 2012 that I emphasized even more in 2013. During the first week, I am very explicit about the collaborative, applied, interactive nature of this course. I invite people to explore with me some ways of learning that are, for some people, uncomfortable and new. I also invite people who aren’t up for such exploration at this time to choose a different section or course, with no hard feelings. Along with better development of the curriculum and assessments, I think this tweak goes a long way toward student-course-instructor fit.

One area for growth that was noted a few times in the qualitative data was lack of clarity about learning assessment requirements. In my view, the handouts and LOs I give to students pretty clearly map on to my rubrics and exams. But there’s obviously a disconnect: a small yet larger proportion of students than in any of my other courses report lack of clarity for what to expect from grading. Moreover, my “fairness of evaluations” rating was the only UMI across all my 2012/2013 courses to fall below 4 out of 5. To remedy this disconnect, I have a couple of ideas: First, I will consider giving—up front—my rubrics for all components to the assignment. A couple of students suggested this, and I think it will help. Second, I will consider ways to give more advice for the exams, especially the midterm. One option would be to give a list of the short answer questions, a subset of which will appear on the exam (will this increase learning? If so, great! Will this increase the mean and/or reduce variance? If so, stress!).

[One of the challenges that I face (that’s not exactly popular with students, in my experience) is that attempts to change anything to do with grading run this risk of inflating the mean and/or changing the distribution of grades. Like all faculty in my department, I am bound by departmental requirements to have a mean around 65% and a standard deviation around 14% in all 100 and 200 level courses. Therefore, interventions that increase the mean or shrink the SD present real concerns that force me to confront this reality: any improvement in clarity might simultaneously require an increase in difficulty.]

Quite a few students mentioned enjoying the readings from one book, but finding the book that drew from a sport psychology perspective a bit less helpful and/or easily applied to academic life. Many of the readings will change in 2013/2014, largely because many of the chapters I currently rely on are out-of-date and must be changed anyway. Therefore, it’s a great opportunity to re-think the whole set of readings. I will be making every effort to get the custom course-pack down to one publisher, ideally with less of an emphasis on sport. Given the current offerings I’ve seen, I think that’s possible. I’ll also be thoughtfully considering the length of readings, as a (small) group of students mentioned feeling like there was too much to read.

Overall, in my view and the students’, this offering was the most successful iteration of Psyc 208 to date!

“Writing-to-Learn” in Intro Psych

While flying to the APS convention in Washington DC today, I was perusing a back-issue of the journal Teaching of Psychology. I came across an interesting article that made me consider a new paper-writing option for my intro psych class. In two studies, the researchers asked intro psych students to upload 16 brief (1-2 paragraph) statements that expanded on one concept from a 10-concept shortlist from each chapter. The paragraphs were graded for completion only (just a check to make sure they were topical), and were then coded for topic choice. Students responded better to midterm exam questions on topics they had previously written about than to questions on topics they had not previously written about–even when students had been told specifically which concepts to write about (Study 2).

What I’m finding particularly intriguing about this study is that (a) students seemed to learn the concepts better after writing about them, (b) the task seems pretty simple and straightforward for students (“What I learned about the concept is…” or “An example of the concept in my life is…”), (c) it may promote strong study habits that can be transferred to other topics and courses, and (d) because it’s ungraded and tracked online, it seems logistically manageable in a very large class.

Indeed, my classes are pretty large (depending on the year/term, they range from 250 to 400 students). Couple that with very limited TA resources, and fitting in writing is always a challenge. Currently I assign one 600-word application-style paper in each term. Recently I added a peer feedback step using peerScholar software, and that worked out pretty well. Informally, students reported learning from reading others’ papers, yet few seemed to revise their work at all, let alone implementing changes based on others’ feedback.

Maybe the assignment in this study could be a worthwhile pursuit. In this research paper, the authors noted that completion of all 16 was worth 10% of the students’ grade. I’m not sure I could offer that just for completion… we have pretty strict grading standards here in UBC Psych. But once I cut it in half (separating term 1 101 from  term 2 102) that’s only 5%… that seems more manageable.

I continue to ponder… as I lie awake… on Pacific time… in Eastern timezone.

Responding to Student Evals 2011/2012 Part 4: Psyc 218 Statistics and Conclusion

Welcome to part four–the final installment!–of my reflections on student evaluations of teaching from 2011/2012. Please see my earlier posts for a general introduction and reflection on feedback from my Psyc 217 research methods, Psyc 100 intro course, and Psyc 208 Section 002 Special Topics course. I have also posted graphs that facilitate comparison across all my courses and years I have taught them.

First, as always, I would like to thank each of my students who took the time to complete a student evaluation of teaching this year. I value hearing from each of you, and every year your feedback helps me to become a better teacher. Please note that with respect to the open-ended responses, I appreciate and consider every thoughtful comment. The ones I write about are typically those that reflect common themes echoed by numerous students.

Psyc 218: Analysis of Behavioural Data

Because I have only taught this course once, I am interpreting the numerical data in reference to my other courses that I have taught multiple times. As you’ll see from the graph below (click on it to enlarge), students rated this course right on par with my others. In fact, ratings were almost exactly what I received from students in my research methods courses. Given that the third midterm was much more challenging than I had anticipated, it surprised me somewhat that students rated evaluations as fair as my students in research methods did. At 3.9 there is definitely room for improvement there (in both courses). Interestingly, clarity of expectations was also high, which lends support for my hypothesis that these two are related (see further discussion in my research methods reflection). Overall, these numbers signal to me that students are feeling positively toward this course.

After reading the comments, I must say an extra thank you to each of you for the polite and thoughtful tone used in delivering this feedback. There most common point of discussion was acknowledgments that the first two midterms were too easy, and the last too difficult and too lengthy. I absolutely agree — it was clear during the semester and is clear in the evaluations. I will make every attempt to even out the difficulty of these exams next time. What I appreciated most was the way these comments were delivered. Here’s an example:

The course was a lot of fun, and easy to understand. However, for the future, I would prefer if we can have midterms with a consistent level. It was a huge shock for me on the last midterm. 🙁

In case you’re writing these kinds of comments in the future, here’s why I found this comment particularly effective: it starts with a positive that was at least a bit specific (could be moreso), and conveys a respectful tone set by phrases like “I would prefer”. There’s no personal attack toward me here, but a fair acknowledgment of an area in this course that needs improvement.

The midterms were by far the most discussed aspect of the course. However, many people also noted how much they valued my enthusiasm and interactive style. Here’s a couple of specific comments that capture the sentiments echoed by many students in the class:

Not one minute is wasted in class. She is always teaching in innovating and varied ways.

Although still challenging, I found this course to be enjoyable. Dr. Rawn was approachable and tried hard to interest and even engage us in the material and provided encouragement to students. I liked how the iclickers were used so that u could test yourself to see if you understood the material without having to worry about losing marks. As already discussed in class, the midterms although sometimes too long or short, I thought the questions were fair (they tested for understanding rather than memorization). I thought the spss assignments were a good way of understanding theory and application of the tests we learned although I found some questions unclear.

For a few people, it seems I was able to calm some of their hesitation toward math/numbers. This is fabulous to hear, because it’s something I really tried to do throughout the course. Here’s an example of one of these comments:

I thought Dr. Rawn did an amazing job teaching this course. I have struggled with math and was not enthusiastic to take this course however, she inspired motivation and made this course interesting and easy to learn. I ended up with a higher grade in this course than I anticipated and I owe it all to Dr. Rawn.

While I’m not convinced this student “owes it all” to me, it seems that I was able to offer some support beyond the technicalities of course content, reflecting one of my personal goals for this course this year. Again, I’d like to thank everyone who completed the student evaluations for doing so in such a thoughtful and respectful way. Your feedback will influence the way I teach this course in the future.

Conclusion

What a helpful exercise that was! Writing about my student evaluations of teaching helped me to really think about what you (my students) were saying about my teaching and my courses. If I had to pick one overall goal for me to keep in mind next year, it would be having clear expectations and communicating those effectively. I think I’m doing this well in some courses, making progress in others, and have more room for improvement in others.

Reading student evaluations can be a very emotional experience for those of us who dedicate our lives to helping others learn. Overall, I’m thankful for the respectful tone that most of my students used when identifying strengths and areas for growth. When feedback isn’t conveyed respectfully it makes it difficult to hear what is being said. Thank you for taking the time to provide feedback, and for the way you did it.

For educators who might be reading this, I also gained insight into a process that helped me be optimally receptive to feedback: (1) I started with the numbers — comparing within year and within course (using graphs) really helped me set up an analytic frame of mind; (2) once I was in that analytic (rather than emotional) frame of mind, I read the comments and used them to help me understand what I was seeing in the numbers; (3) I wrote about it–you might share it or not, but this was really helpful for me to make sure I processed the messages and decided on action plans.

Onward and upward!