Two-Stage Exams for English-Language Learners

Twice a year, I travel to China to teach intensive 2-week forestry courses (in English) to Chinese undergraduates. Students face several challenges in this learning environment, but English-language ability is the biggest. (I’m not criticizing the students’ abilities. It is simply a key component of the learning environment. I have utmost respect for students taking a course taught in a foreign language.) Students in my courses have a wide range of English abilities. For students with decent English skills, learning in a foreign language introduces extraneous cognitive load (translation efforts use some cognitive bandwidth, leaving less available for learning). Unfortunately, students with marginal English skills simply miss or misunderstand much of what I say when I teach.

I have a few strategies regarding language (to be detailed in a future post), but the newest is the use of two-stage exams (more details here). In a two-stage exam, students complete an individual exam, hand it in, and then complete an identical or similar exam in small groups. I weight the exam scores 85% individual vs. 15% group. This is just enough to give incentive for the group portion, but I view that stage as a learning exercise rather than a test. Students are highly engaged in the group portion as they debate and explain answers within their group.

The group stage nicely incorporates a few techniques or practices that have been shown to benefit learning.

Peer Instruction and Experts vs. Novices

When a student explains a concept or idea to another student, this is peer instruction. Assuming the first student correctly understands the answer, this can overcome issues inherent to experts teaching novices. I am (hopefully) an expert on the content that I teach, and I learned it long ago. Therefore, I innately know the steps in solving a problem or the connections among topics and ideas, perhaps without even being able to elucidate them (NRC 2000). This is unconscious competence (Sprague and Stuart, 2000). I may be a poor explainer of a topic because I will skip steps or fail to point out connections that are so obvious to me that I don’t even think of them consciously. On the other hand, a student who has just mastered an idea possesses conscious competence. They know what the crux of the idea is, they can list all the steps and connections. They may be able to explain it in a manner that is clearer to their peers.

Self-Explanation

I’ve already posted about the two most effective learning strategies, retrieval practice and distributed practice. In a review of learning strategies by Dunlosky et al. (2013), self-explanation ranked in the next most effective category. In self-explanation, a student verbally explains the steps to a problem and the principles behind those steps. This results in improved learning (Chi et al. 1994).

Language

Peer instruction and self-explanation are relevant here regardless of the language of instruction. When it first occurred to me to try two-stage exams during my teaching in China, it was because I started using the approach this semester at UBC, and I’ve been pleased with the results. Then the real magic occurred to me. Though I instruct the students in English (as the program is intended to work), a sizeable minority clearly miss quite of bit of what I say. I have a Chinese teaching assistant who translates some complex ideas as necessary, but it’s not feasible to translate everything. In the group portion of the exam, the students speak their native Mandarin, even when they write the answers in English. This means even students with weak English skills are involved in a motivated discussion of each exam question, which is hopefully led by a student with stronger English, and therefore a better understanding of the concept in question. The benefits these students receive from peer instruction are greatly compounded by the fact that it occurs in their native language.

Some Results

At the end of the two week course, I asked the students if they thought the two-stage exams improved their learning, and they all said yes. (I asked for honest feedback, though I’m sure some felt some pressure to say yes, so I’ll round the unanimous agreement down to a strong majority). On to the numbers…

On both exams, students had a respectable improvement in scores during the group exam. This raises the question of whether the groups possess synergy, or if the strongest group member is just bringing the group score up to their level. Of the five groups, one group in each exam received a slightly lower score in the group portion compared to the highest individual score of the group members.

In the other four groups, the group score was higher than all of that group’s individual scores. On average, groups scores were 7.5% higher (exam 1) and 11.2% higher (final exam) than the highest individual score from that group. This shows that (in 4/5 groups), even the strongest member of the group was benefiting from the knowledge of their lower-performing group mates. These score improvements may not directly correspond to gains in knowledge, but my hope and expectation is that students did gain a better understanding of the material during the group portion of the exam.

Considerations

One challenge of a two-stage exam is that the exam needs to be shortened to fit both stages in the regular exam time. My schedule for this course involved a ~3-hour class period on exam days. I had no intention to give a 3-hour individual exam, and it would not have been fair to give a lecture immediately followed by an exam. This time slot was just right for a 1.75-hour exam, a short break, and then a 45-minute group portion. In two-stage exams I’ve given so far, students tend to complete the group portion in 40%-50% of the time it takes them to do the (identical) individual portion.

References