e-valuation

The Flexner Report of 1910 was an evaluation of medical education in all programs in Canada and the US–Flexner found medical education wanting and his report lead to significant changes in medical education. But for evaluators, the importance of this report is that it was the genesis of accreditation, a model of evaluation based on expert, professional judgement. The New England Journal of Medicine has published a 100 year retrospective look at medical education and still finds it wanting.

Accreditation has changed over the years, moving away from looking at resources and processes, to a more substantial examination of outcomes. But still, the approach holds too fast to simplistic criteria, and still often looks at things like the ration of support staff to faculty, and the number of hours of clinical work, and the office space allocated. And it is common knowledge that accreditation is as much about feting the site visitors as it is about doing a good job of professional education.

There have been few advances in accreditation as evaluation, perhaps the one exception is TEAC or the Teacher Education Accreditation Council. Unlike other accrediting agencies, TEAC holds programs accountable for what they say they intend to do based on reasonable evidence identified by the program being reviewed, rather than an abstract, general notion of what is considered appropriate professional education and preconceived sorts of evidence.

Now, if only someone would do a 100 year retrospective analysis of accreditation qua evaluation…

Everything that can be counted does not necessarily count; everything that counts cannot necessarily be counted. Albert Einstein

Not life, but good life, is to be chiefly valued. Socrates

True genius resides in the capacity for evaluation of uncertain, hazardous, and conflicting information. Winston Churchill

And there’s a mighty judgement coming, but I may be wrong. Leonard Cohen

I have added a new category to the blog, and will be posting quotes about evaluation… The first is one of my absolute favorites…

We cannot discover what ought to be the case by examining what is the case. We must decide what ought to be the case. Paul Taylor

Educational and psychological tests have pockets of darkness in their history. A key one is eugenics, the continued connections between mental measurement and racism. The unbelievably goofy pseudo-scientist J. Phillipe Rushton is featured in my home town newspaper, the Vancouver Sun, for his silly study in which he asserts scores on the SAT demonstrate men are naturally smarter than women. Apparently no one told him the SAT doesn’t even predict success in college, let alone success in life. Apparently he is also unaware that even the test publisher says the test isn’t worth much, especially in judgements of merit, when a point spread of at least 125 is necessary to even suggest real differences exist. Rushton can’t even keep his silly science straight – he previously claimed there is an inverse relationship between intelligence and penis size. Seems to me the women have it all over the men based on this theory.

As part of Emory University’s Series on Excellence, I gave a talk on excellence in evaluation. You can listen to this presentation on the Emory website.