{"id":2110,"date":"2017-04-16T04:22:09","date_gmt":"2017-04-16T12:22:09","guid":{"rendered":"https:\/\/blogs.ubc.ca\/iiqbal\/?p=2110"},"modified":"2017-08-20T04:30:33","modified_gmt":"2017-08-20T12:30:33","slug":"course-design-intensive-evaluation","status":"publish","type":"post","link":"https:\/\/blogs.ubc.ca\/iiqbal\/2017\/04\/16\/course-design-intensive-evaluation\/","title":{"rendered":"Evaluating the Course Design Intensive"},"content":{"rendered":"

\"Rainy<\/a><\/p>\n

As do several other teaching and learning centres, ours offers a Course Design Intensive<\/a>\u00a0(CDI). During this 3-day course, participants make progress on the design or redesign a course for post-secondary students.<\/p>\n

Since 2015, I have been leading a\u00a0program evaluation of our CDI. The process and methodology have been messy and inconsistent…and have taught me a lot about program evaluation. In this blog post, I share on the\u00a0retrospective pre-test <\/strong>(RPT), one of the approaches I have used as part of our multi-faceted evaluation [for a 2-page description of our program evaluation, see here<\/a>].<\/p>\n

Retrospective pre-test<\/h4>\n

The retrospective pre-test is a survey that is\u00a0administered at the same time as the post-test. Learners are asked to answer questions about their level of understanding, confidence or skill after<\/em> an intervention. They are then asked to think back to their understanding prior<\/em> to the intervention and to answer the same questions, but from the perspective of the present moment. See here<\/a> for more information, including a\u00a0brief description of strengths and weaknesses of this approach.<\/p>\n

What we used to do before<\/h4>\n

Prior to December 2016, we\u00a0did the following:<\/p>\n

Before the CDI<\/strong><\/p>\n

The survey\u00a0asked\u00a0participants to\u00a0consider the learning outcomes for the CDI<\/a> and, using a Likert Scale, rate: (1) how important is this skill in course design?; (2) how confident are you in your current skills in this area? [see here for pre-CDI program evaluation survey<\/a>].<\/p>\n

On day 3 of the CDI<\/strong><\/p>\n

On the last day of the CDI, participants would complete a survey\u00a0that had the same questions as above (#1 and #2) and this question: (3) how helpful has the CDI been in learning this skill? [see here for post-CDI program evaluation survey<\/a>]<\/p>\n

What we do now: Retrospective pre-test and post-test<\/h4>\n

Instead of administering two surveys at two different times, we now administer the retrospective pre-test and post-test at the same time. After consulting different articles about the benefits and disadvantages of one method over another, I surmised that the main advantage, in the case of the CDI, was mostly practical: one survey vs two. \u00a0To access our survey, see here<\/a>.<\/p>\n

Additional resources on the retrospective pre-test<\/h5>\n

The Retrospective Pretest: An Imperfect but Useful Tool<\/a>\u00a0(Harvard Family Project, 2005)<\/p>\n

The Retrospective Pretest Method for Evaluating Training<\/a>\u00a0(Evaluate Webinar, 2015)<\/p>\n

Program Development and Evaluation<\/a>\u00a0(University of Wisconsin-Extension<\/a>) Go to Education in Program Development<\/em> tab for various resources on the retrospective pre-test.<\/div>\n

 <\/p>\n

Notes:<\/p>\n

    \n
  1. Thanks go to Dr. Chris Lovato<\/a> for introducing me to\u00a0retrospective pre-test.<\/li>\n
  2. Photo credit:\u00a0Bill Dickinson<\/a>\u00a0“Rainy Days and Mondays”\u00a0https :\/\/flic.kr\/p\/oMKxVd<\/li>\n<\/ol>\n

     <\/p>\n

     <\/p>\n

     <\/p>\n","protected":false},"excerpt":{"rendered":"

    As do several other teaching and learning centres, ours offers a Course Design Intensive\u00a0(CDI). During this 3-day course, participants make progress on the design or redesign a course for post-secondary students. Since 2015, I have been leading a\u00a0program evaluation of our CDI. The process and methodology have been messy and inconsistent…and have taught me a […]<\/p>\n","protected":false},"author":792,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[60,1484100],"tags":[296934,1484100,2113092],"class_list":["post-2110","post","type-post","status-publish","format-standard","hentry","category-blog","category-educational-development","tag-course-design","tag-educational-development","tag-program-evaluation"],"_links":{"self":[{"href":"https:\/\/blogs.ubc.ca\/iiqbal\/wp-json\/wp\/v2\/posts\/2110","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.ubc.ca\/iiqbal\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.ubc.ca\/iiqbal\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.ubc.ca\/iiqbal\/wp-json\/wp\/v2\/users\/792"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.ubc.ca\/iiqbal\/wp-json\/wp\/v2\/comments?post=2110"}],"version-history":[{"count":11,"href":"https:\/\/blogs.ubc.ca\/iiqbal\/wp-json\/wp\/v2\/posts\/2110\/revisions"}],"predecessor-version":[{"id":2168,"href":"https:\/\/blogs.ubc.ca\/iiqbal\/wp-json\/wp\/v2\/posts\/2110\/revisions\/2168"}],"wp:attachment":[{"href":"https:\/\/blogs.ubc.ca\/iiqbal\/wp-json\/wp\/v2\/media?parent=2110"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.ubc.ca\/iiqbal\/wp-json\/wp\/v2\/categories?post=2110"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.ubc.ca\/iiqbal\/wp-json\/wp\/v2\/tags?post=2110"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}