In this op ed piece, Francis Schrag points out a key feature of evaluation by comparing an evaluation of NCLB and the diabetes drug Avandia. A key element that must be included in any evaluation are side effects, the unanticipated outcomes. Even if the planned outcomes occur and in large measure they can be nullified by the presence of harmful unanticipated outcomes…’No Child Left Behind’ doesn’t provide full picture
Francis Schrag
Guest columnist — 6/13/2007
Newspaper readers may have noticed recent articles reporting test score performance of Wisconsin or Madison public school students as well as articles reporting controversy surrounding the diabetes drug Avandia. It’s illuminating to compare the two.
In the latter case, there is apparently strong evidence that Avandia is effective — it lowers the level of sugar in the blood. This fact, however, does not automatically lead to endorsement of the drug. Why not? Because as many now know, there are potential safety concerns, notably alleged increased risk of heart attack.
There is an important lesson here in the medical sphere that ought to carry over to the educational sphere: Efficacy is not all we care about.
Just as all drugs produce multiple effects, so do all education policies, such as the No Child Left Behind law passed in 2002. It is tempting to assess the impact of such a law simply by comparing test scores in math and reading (the two subjects where annual testing is authorized) before and after passage of the law. After all, test scores reflect student achievement, and that is presumably what we’re after.
Alas, this comparison, difficult enough to make for all sorts of reasons, is the equivalent of measuring blood-sugar levels before and after use of Avandia without taking into account any side effects.
Which side effects should be taken into account? It would be nice to evaluate many, but evaluation is costly and time-consuming, so let’s restrict ourselves to one that is significant: continuing motivation to learn in all subjects. Why this one? Because, just as the increased risk of heart attack may outweigh the beneficial effects of a diabetes drug, so might a reduction in continuing motivation outweigh a modest gain in achievement scores.
How could we assess continuing motivation? First, we need to compare the motivation of public school students subject to the law with matched private school students who are not. Second, we need to provide both groups of students with opportunities to manifest continuing interest in learning by giving them the option to participate in an activity that would manifest that interest, for example reading additional books over the summer or participating in an after-school science fair.
Evaluators will need plenty of imagination to come up with valid ways of tapping students’ motivation to learn. This may be difficult, but failing to consider important side effects of school learning is irresponsible. Without a conscientious effort to tap important side effects, we’ll have no basis for ruling out the possibility that a policy designed to raise test scores does so only by putting another valued outcome at risk.
We don’t want our educational policies to be the equivalent of Avandia, but so far we’re making no effort to find out if they are.
Francis Schrag is a professor emeritus in the Department of Educational Policy Studies at the University of Wisconsin-Madison.
Follow
