Tag Archives: course design

Evaluating the Course Design Intensive

Rainy Days and Mondays

As do several other teaching and learning centres, ours offers a Course Design Intensive (CDI). During this 3-day course, participants make progress on the design or redesign a course for post-secondary students.

Since 2015, I have been leading a program evaluation of our CDI. The process and methodology have been messy and inconsistent…and have taught me a lot about program evaluation. In this blog post, I share on the retrospective pre-test (RPT), one of the approaches I have used as part of our multi-faceted evaluation [for a 2-page description of our program evaluation, see here].

Retrospective pre-test

The retrospective pre-test is a survey that is administered at the same time as the post-test. Learners are asked to answer questions about their level of understanding, confidence or skill after an intervention. They are then asked to think back to their understanding prior to the intervention and to answer the same questions, but from the perspective of the present moment. See here for more information, including a brief description of strengths and weaknesses of this approach.

What we used to do before

Prior to December 2016, we did the following:

Before the CDI

The survey asked participants to consider the learning outcomes for the CDI and, using a Likert Scale, rate: (1) how important is this skill in course design?; (2) how confident are you in your current skills in this area? [see here for pre-CDI program evaluation survey].

On day 3 of the CDI

On the last day of the CDI, participants would complete a survey that had the same questions as above (#1 and #2) and this question: (3) how helpful has the CDI been in learning this skill? [see here for post-CDI program evaluation survey]

What we do now: Retrospective pre-test and post-test

Instead of administering two surveys at two different times, we now administer the retrospective pre-test and post-test at the same time. After consulting different articles about the benefits and disadvantages of one method over another, I surmised that the main advantage, in the case of the CDI, was mostly practical: one survey vs two.  To access our survey, see here.

Additional resources on the retrospective pre-test

The Retrospective Pretest: An Imperfect but Useful Tool (Harvard Family Project, 2005)

The Retrospective Pretest Method for Evaluating Training (Evaluate Webinar, 2015)

 

Notes:

  1. Thanks go to Dr. Chris Lovato for introducing me to retrospective pre-test.
  2. Photo credit: Bill Dickinson “Rainy Days and Mondays” https ://flic.kr/p/oMKxVd

 

 

 

Big ideas and course design

 

“A big idea must have pedagogical power: It must enable the learner to make sense of what has come before; and, most notably, be helpful in making new, unfamiliar ideas seem more familiar….a big idea is not just another fact or a vague abstraction but a conceptual tool for sharpening thinking, connecting discrepant pieces of knowledge, and equipping learners for transferable applications.”

(Wiggins & McTighe, 2005, p.70)

dandelion

Below are some notes on the concept of “Big Ideas,” as presented in Understanding by Design. This information is part of the Course Design Intensive, a 3-day workshop for university instructors seeking to design or re-design a course.

Big ideas are at the core of a subject/field. They are often abstract, non-obvious, and counterintuitive to the novice (Note 1). Big ideas are essential for making coherent connections in a field and are a conceptual anchor that makes facts more understandable and useful (p.80).

A big idea can manifest in various formats (phrase, words, question etc). In pedagogical practice, a big idea may appear as a helpful:

  • concept (e.g. adaptation, perspective)
  • theme (e.g. “coming of age”)
  • on-going debate and point of view (e.g. nature versus nurture, conservatives vs liberals)
  • paradox (e.g. freedom must have limits)
  • theory (e.g. evolution vs natural selection, social constructivism)
  • underlying assumption (e.g. markets are rational)
  • recurring question (e.g. “Can we provide it?” “Is that fair?”) 
  • understanding or principles (e.g. correlation does not ensure causality, the reader has to question the text to understand it)  (p.70)

From the above examples, we can see that big ideas are:

  • broad and abstract
  • represented by few words
  • universal in application
  • timeless (p.69)

In summary, a big idea:

  • provides a conceptual lens
  • provides breadth of meaning by connecting and organizing many facts, skills and experiences
  • points to ideas that are at the heart of expert understanding of the subject/field
  • requires “uncoverage” because its meaning or value is rarely obvious to learner
  • applies to many other inquiries and issues over time (great transfer value) (p.69)

 

 

Note 1: Wiggins & McTighe distinguish big ideas from basic ideas. The latter, they write, are “the basis for further work; for example, definitions, building-block skills, and rules of thumb.” (p.67).

Basic term Core idea
Graph “Best fit” curve of the data
Ecosystem Natural selection
Fact versus opinion Credible thesis

 

Photo credit: Stephanie Carter, “dandelion” https ://flic.kr/p/349d57