As do several other teaching and learning centres, ours offers a Course Design Intensive (CDI). During this 3-day course, participants make progress on the design or redesign a course for post-secondary students.
Since 2015, I have been leading a program evaluation of our CDI. The process and methodology have been messy and inconsistent…and have taught me a lot about program evaluation. In this blog post, I share on the retrospective pre-test (RPT), one of the approaches I have used as part of our multi-faceted evaluation [for a 2-page description of our program evaluation, see here].
Retrospective pre-test
The retrospective pre-test is a survey that is administered at the same time as the post-test. Learners are asked to answer questions about their level of understanding, confidence or skill after an intervention. They are then asked to think back to their understanding prior to the intervention and to answer the same questions, but from the perspective of the present moment. See here for more information, including a brief description of strengths and weaknesses of this approach.
What we used to do before
Prior to December 2016, we did the following:
Before the CDI
The survey asked participants to consider the learning outcomes for the CDI and, using a Likert Scale, rate: (1) how important is this skill in course design?; (2) how confident are you in your current skills in this area? [see here for pre-CDI program evaluation survey].
On day 3 of the CDI
On the last day of the CDI, participants would complete a survey that had the same questions as above (#1 and #2) and this question: (3) how helpful has the CDI been in learning this skill? [see here for post-CDI program evaluation survey]
What we do now: Retrospective pre-test and post-test
Instead of administering two surveys at two different times, we now administer the retrospective pre-test and post-test at the same time. After consulting different articles about the benefits and disadvantages of one method over another, I surmised that the main advantage, in the case of the CDI, was mostly practical: one survey vs two. To access our survey, see here.
Additional resources on the retrospective pre-test
The Retrospective Pretest: An Imperfect but Useful Tool (Harvard Family Project, 2005)
The Retrospective Pretest Method for Evaluating Training (Evaluate Webinar, 2015)
Notes:
- Thanks go to Dr. Chris Lovato for introducing me to retrospective pre-test.
- Photo credit: Bill Dickinson “Rainy Days and Mondays” https ://flic.kr/p/oMKxVd