Tag Archives: value added measures

Why paying teachers for student test scores is a bad idea

While everyone in British Columbia is paying attention to the teachers’ strike, the Fraser Institute launched its latest effort to marketize education. This week the Fraser Institute, a neoliberal think tank, released a report promoting incentivized pay for teachers.

Teacher Incentive Pay That Works, summarizes 10 “case studies” from around the globe, which the Fraser Institute argues illustrates successful incentive pay programs. The press release for the report is titled “Evidence shows teacher incentive pay improves student performance,” which is ironic since the report ignores the long history of these schemes, and studiously avoids the details of the debate around value-added measurements in the United States (which is currently enthralled in a public revolt against test-driven education), as well as evidence illustrating damage done to schools and learning under such schemes.

The Illusion of Paying Teachers for Performance

… history shows that any pay-for-performance gains are mostly illusions. Not only do they fail to improve student achievement, they are also destructive, encouraging administrators and teachers to cheat by manipulating statistics, or by teaching to the test. Inevitably, children wind up the losers because curricula are narrowed to include subjects that can be taught by drill and repetition and that are easily measured. (Wilms & Chapleau, 1999)

Wilms and Chapleau note that pay-for-performance was first rolled out in England, around 1710! Teachers’ salaries were based on their students’ scores on examinations in the “three ‘Rs.” “This early payment-for-results system had great appeal because it promised to help keep children from poor families in school, where they might learn the basics.”

The scheme became a permanent fixture in English schools by 1862 (as part of the Revised Education Code) and was in effect for over 30 years. Historical accounts of England’s scheme describe teachers and administrators as becoming obsessed with the systems financial rewards, which according to Wilms and Chapleau were dubbed “the cult of the [cash] register.”

Curriculum was narrowed to include just the easily measured basics. The sciences and the arts, along with many other non-tested activities disappeared from schools (foreshadowing the disappearance of recess from elementary schools in the United States as a result of the test driven reforms like Obama’s Race To The Top).

Teaching became increasingly mechanical, as teachers found that drill and rote repetition produced the “best” results. One schools inspector wrote an account of children reading flawlessly for him while holding their books upside down.

The English system of pay for performance produced a mechanical approach to teaching and learning that eroded teacher creativity. Standards for student success (or failure) were spelled out in detail (just as the new Fraser Institute reports as a “Key Lesson 1” in their study, “Define what we expect teachers to do.”)

An inspector wrote that the Education Code “did all the thinking for the teacher; it told him in precise detail what he was to do each year.” Another recalled, “Every teacher in the country takes his orders from the Code, studies the Code, and devotes his energies to satisfy or to circumvent it.”

Predictably the English system imploded in a cheating scandal that included falsification of records and teachers coaching student through examinations, not unlike the recent massive cheating scandal in Atlanta, Georgia and across the USA, which highlights deleterious effects of test-driven education.

the overwhelming judgment was that it was unsound policy. Cynics referred to schools as “grant factories” and children as “grant-earning units.”

In the later third of the 19th Century, teacher pay-by-results appeared briefly in Canada. Student achievement initially rose but, as in England, teachers started to focus on students who were most likely to succeed, turning their classrooms into test prep centres. By 1883 the Canadian experiment ended as a result of public outrage.

One hundred years later in the United States, the Nixon administration funded an experiment in “performance contracting” in which school funding was tied to students standardized test scores. The experiment provided incentives for administrators, teachers, and students. Private contractors, who were suppose to bring innovation and business know-how to the effort, were given contracts in 18 cities to raise student achievement levels in reading and math.

Turns out contractors offered no pedagogical innovations only teaching to the test. The project was declared a failure in the midst of poor results and a cheating scandal.

As Wilms and Chapleau illustrate, the wake of pay-for-results education reforms is strewn with detritus of dishonest behaviour (cheating, falsifying records) and teaching to the test.

Similar incentive efforts in the 1990s and the recent examples of cheating scandals in Atlanta and Texas prove that incentive pay reform is a failed idea.

Flawed Logic of Performance Pay

Donald Gratz, the author of Perils and Promise of Performance Pay, describes the flawed logic of incentive pay plans that aim to boost student achievement.

False assumptions #1: Teachers lack motivation.

Teachers care about their students and want them to succeed. “Does anyone really think that large numbers of teachers know what their students need but are willfully withholding it? That they would help students learn more, if only someone offered them a bonus to do so? This is a highly cynical view of teachers, one that teachers understandably find demeaning, not motivational.”

False Assumption #2: Schools are Failing

The manufactured crisis of school failure is a basis for corporate education reform or what is also called the Global Education Reform Movement (Berliner & Biddle, 1995). This is not to say that there aren’t troubled schools or that public schools do not need to be improved, but most students have higher levels of academic achievement now than in the past.

False Assumption #3: Measuring Academic Achievement is All that Counts

“If we want students to develop as well-rounded human beings who are empathetic, thoughtful, and creative, we will have to include these characteristics among our goals for schools and seek ways to gauge our success. A system that rewards schools, students, and teachers only for test scores will get mostly test scores. This is not what most of us want for our children.”

And What About the Research on Incentive Pay?

The National Center for Fair and Open Testing has compiled information surveying the research on paying teaching for test scores and concludes that it is a practice that damages schools and undermines learning.

Paying for higher test scores creates score inflation, not genuine learning. Paying for test scores encourages teaching to the test, which creates inflated results without improving learning. (Koretz, 2009; Madaus, Ressell & Higgins, 2009; Nichols & Berliner, 2007)

Payment for performance narrows the curriculum to what is tested and leads to reduced focus on or elimination of important subjects, such as social studies, science, art, music, and physical education. (McMurrer, 2007; Morton & Dalton, 2007)

It is unfair and ineffective to pay teachers for test results that are often marred by scoring and other errors. (Rhoades & Madaus, 2003).

Payment for gains in student scores does not solve the problem of test-induced educational damage. There are too many flaws in “value-added” measurement approaches to trust the results. (McCaffrey, et al., 2005; Bracey, 2007; National Research Council, 2009)

Most teachers’ primary motivation is not high pay. If it were, they would have chosen another profession. Teachers know test scores are a poor barometer of their abilities, so pay for performance damages rather than enhances their sense of professionalism and morale (Whitford & Jones, 2000; Nichols & Berliner, 2007). It can decrease motivation (Ryan & LaGuardia, 1999). Payment for “performance” also has been shown to increase cheating (Pfeffer, 2007).

Payment for test scores may not even to raise student scores and has been shown in one country to reduce scores. This is despite the extensive evidence of score inflation from teaching the test (Martins, 2009; Springer, Podgursky, & Lewis, 2009).

Paying individual teachers for student scores encourages unhealthy competition. Incentive pay may reduce cooperation among teachers and can cause divisions among staff and parents (MacInnis, 2009; Pfeffer, 2007). In addition the OECD has recently released a report that says competition in education is a failed policy. The bottom-line:

Research on pay for performance finds that it rests on dubious assumptions and lacks evidence it succeeds, and there is good evidence that it often fails.

References

Berliner, D. C., & Biddle, B. J. (1995). The manufactured crisis: Myths, fraud, and the attack on America’s public schools. New York: Basic Books.

Bracey, J. 2007. Evaluating value added. FairTest Examiner, July. http://www.fairtest.org/whats-value-growth- measures

Bradshaw, W. J., & Gallup, A. M. (2008, September). Americans speak out: Are educators and policy makers listening? Phi Delta Kappan, 90(10), 7–31.

Gratz, D. B. (2009). Perils and promise of performance pay. Lanham, MD: Rowman and Littlefield.

Koretz, D. (2009, April 29). What’s Missing in Obama’s Education Plan? Education Week. http://www.edweek.org/ew/articles/2009/04/29/30koretz_ep.h28.html?tkn=QTLFEqyaUfgkzI4vRyp6Q0c2kzhDTpngNM 9B&print=1

MacInnes, G. (2009). Eight reasons not to tie teacher pay to standardized test results. Century Foundation Issue Brief. http://www.tcf.org/publications/education/gordon%20brief.pdf

Madaus, G., Russell, M., & Higgins, J. (2009). The Paradoxes of high stakes testing. Charlotte, NC: Information Age Press.

Martins, P. (2009, March). Individual teacher incentives, student achievement and grade inflation. Queen Mary, University of London, CEG-IST and IZA, Discussion Paper No. 4051.

McCaffrey, D., Koretz, D., Lockwood, J.R., & Hamilton, L. (2005). Evaluating value-added models for teacher accountability. Santa Monica: RAND Corporation.

McMurrer, J. (2007). Choices, changes, and challenges: Curriculum and instruction in the NCLB Era. Center on Education Policy. http://www.cep-dc.org/

Morton, B. & Dalton, B. (2007). Changes in instructional hours in four subjects by public school teachers of grades 1 through 4 (Issue Brief). National Center for Education Statistics. http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2007305

National Research Council, Board on Testing and Assessment. (2009). Letter Report to the U.S. Department of Education on the Race to the Top Fund. National Academy of Sciences, available at http://www.nap.edu/catalog.php?record_id=12780

Nichols, S.L, & Berliner, D.C. (2007). Collateral damage: How high-stakes testing corrupts America’s schools. Cambridge: Harvard Education Press.

OECD. (2014). When is competition between schools beneficial? PISA in focus, 42. http://www.oecd.org/pisa/pisaproducts/pisainfocus/PISA%20in%20Focus%20N42%20(eng)–Final.pdf

Pfeffer, J. (2007). Testimony to the U.S. House of Representatives. http://federalworkforce.oversight.house.gov/documents/20070313111150-45256.pdf

Rhoades, K. & Madaus, G., (2003). Errors in standardized tests: A systemic problem. Boston College. http://www.bc.edu/nbetpp

Ryan, R. M., & La Guardia, J. G. (1999). Achievement motivation within a pressured society: Intrinsic and extrinsic motivations to learn and the politics of school reform. In T. Urdan (Ed.) Advances in motivation and achievement (Vol 11). Greenwich, CT: JAI Press.

Springer, M., Podgursky, M., & Lewis, J. (2009). Texas Educator Excellence Grant (TEEG) program: Year two evaluation report. http://www.performanceincentives.org/ncpi_publications/policybriefs.asp

Whitford, B. L., & Jones, K. (2000). Accountability, assessment, and teacher commitment. Albany: SUNY Press.

Wilms, W. W., & Chapleau, R. R. (1999, November 3). The illusion of paying teachers for student performance. Education Week, 19(10), 34, 48.