Peer Review of papers in large classes

In an introduction to Philosophy course in Spring 2010, which had around 120 students and six discussion sections, I did several experiments. One was with having students do a peer evaluation of each others’ papers. This was pretty tricky in such a large class. It was not as successful as I had hoped, mostly because of the logistical difficulties.
In each discussion section, which had between 15 and 25 students, I broke the students up into small groups of 4-5 students each. I did this randomly, before the class began. They met with their small groups most weeks, as they were also doing presentations to their small groups. After they turned in their first and second papers, we did a peer review exercise: I had them trade their paper with one other person in their small group. If there were four people, it was easy: two pairs of people traded with each other. If there were three, they just passed the paper one person to the left. If there were five, then two people traded and three people did the pass the paper to the left thing. These papers were the same ones they had already turned in for a mark–they were not getting comments on drafts.

The students had one week to read the paper they were assigned to read, and write down at least two comments. The following week in discussion section we devoted 10-20 minutes or so to trading comments on the papers. I just went around to make sure that each person had written down at least two comments, and I listened in to the conversations as much as I could by moving around between the groups.

Benefits: students get a chance to read other students’ written work, which can show them what might work and what might not work well in writing. They get a chance to think about the material perhaps in a new way than they had before.

Drawbacks: (a) I didn’t have time to find out the quality of the peer review comments–I don’t know how good they were, or how helpful they were to the students themselves. It is possible that students may have made some comments I wouldn’t have agreed with, and thus steered their fellow students in a problematic direction. (b) Logistics: students needed to be in class for the day that papers were traded. I hoped to mitigate this potential problem by having the papers due to be turned in at the same time and place, so most students would do their best to be there. But there are always those who don’t turn their work in on time. Getting their papers to their partners was a problem. And getting their partners papers to them, ditto. I suggested posting the papers to the discussion board for the discussion section on the course website (viewable only by members of that discussion section), but not everyone (understandably) was ready to give permission to do this. Most didn’t want to trade email addresses either (understandably), and I don’t have access to all the students’ email addresses so they could pass their papers to me and then I could pass them on to the other students. I could have set up a separate discussion board on the course website just for those pairs of students, and that might have worked.

An alternative: do the peer review the same day the papers are due: have the students read each others’ papers right then and there and comment. Problem with this: it’s hard to really make good comments when you read the paper quickly like this (I know from experience).

How to review the students’ comments on each others’ papers and ensure that they are reasonable and helpful, when one has such a large class?