Category Archives: Educational evaluation

the complexity of teacher evaluation

Good report on the flaw in some current thinking about teacher evaluation from NEPC. The value added approach is a simplistic strategy for determining teaching effectiveness.

Due Diligence and the Evaluation of Teachers
by Derek C. Briggs, Ben Domingue
February 8, 2011

The research on which the Los Angeles Times relied for its August 2010 teacher effectiveness reporting was demonstrably inadequate to support the published rankings. Using the same L.A. Unified School District data and the same methods as the Times, this study probes deeper and finds the earlier research to have serious weaknesses.

Chomsky on Dewey… what might a “good” school look like

The folly of ‘value-added’ teacher evaluation

Gotta love Alfie Kohn, and in this Huffington Post article he clearly illustrates the problems with the value added (meaning increased test scores) approach to evaluating teacher performance. But Kohn’s article also points out common misconceptions in many evaluation contexts, like uniformity is the same as quality and value is easily perverted or narrowed by the indicators selected.

new AFT blog: What Should Count

A new blog from the AFT, What Should Count, that is self described as follows:

The American Federation of Teachers believes that accountability should be about making sure students have resources to learn and succeed: rich curricula, excellent facilities, talented—and well-supported—faculty, and robust academic standards that are devised and improved by the people who deliver them. This website is designed to serve not only as a clearinghouse of accountability initiatives at the international, national, state and local levels, but also as a starting point for discussing accountability systems that best help our students succeed.

Time will tell whether the AFT contributes positively to the discourse on assessment K-16, but they do have some atoning to do, so this may be a positive start. With Albert Shanker as president, the AFT embraced the standards and assessment reform that began with A Nation at Risk and supported testing new teachers. Initially the AFT neither endorsed or opposed NCLB, but Sandra Feldman’s 2003 comments suggest an endorsement:

The federal NCLB Act poses yet another test of our ability to be con- structive, responsive, and creative while simultaneously fighting and protecting against the indefensible. The law is built around goals we’ve long supported: high academic standards and achievement, eradicating achievement gaps between the haves and the have-nots, making sure that every teacher in every school is qualified, and, yes, accountability. The law also mandates reporting outcomes by student subgroup which is the right thing to do because it puts inequities out there for all to see. (Feldman, S. (2003, July). Keynote address to the AFT QuEST (Quality Education Standards in Teaching) conference, Washington, DC.)

AFT’s opposition to test driven reform has, however, been sharper in recent years in response to pressure from its rank and file members. The AFT’s conservatism and strategy of working behind the scenes doesn’t obviously position the teacher union that represents most urban school teachers as a force for change. Current president Randi Weingarten’s testimony before the House Committee on Education and Labor suggests a continued support for national standards, standardization, and using test scores (at least in part) in determining teacher pay.
So, good for the AFT for creating this blog, but here’s hoping they do much more to contribute to a quality work life for their members, and quality education for children living in US cities.

Military Academies… who says educational reform needs to be anything other than ideological?

While military academies have been part of Chicago schools since 1999, the surge in efforts to create more is palpable, and Arne Duncan is a big supporter (no surprise there). The latest development is the financial incentive to local often financially strapped school districts to open charter schools run by the Marine Corps. Little to no evidence suggests that these schools improve the quality of education or academic achievement for students. However, Congress has passed a bill that supports increases in Junior ROTC. In the absence of evidence that such military academies are better schools, one has to conclude that they serve other purposes–fueling the military ranks and using vulnerable students for the benefit of the corporate-military apparatus.

Evaluations of all charter schools, including military academies need to consider a broad context and consider their worth and value by taking into consideration more than the simplistic representations of their value and dig through a state supported rhetoric that is at best diversionary.

http://www.msnbc.msn.com/id/31108509/

When Smart People Evaluate

Michele Lamont in How Professors Think: Inside the Curious World of Academic Judgment opens the Pandora’s Box of peer review, the primary form of evaluation in higher education. Lamont’s curiosity, like Pandora’s, reveals secretive deliberations that all too often amount to judgments of quality based on the similarity of the work being judged to that of the judges. Lamont examines differences across disciplines, highlights the tension between the idea of having independently established criteria and standards and the inevitability of situational deliberation on what is good or bad, and ultimately calls for a more open, transparent approach to evaluation in higher education. In this later move, she searches for the hope that Pandora found at the bottom of the box.

Lamont describes the details of her book in a short essay in the Huffington Post.

Hope and Change in Educational Evaluation?

I have to confess I was disappointed when Shepard Fairey developed the posters for Obama’s campaign. Shepard Fairey, before the Obama campaign, a skateboarder and graffiti artist stood for a challenge to authority, war and capitalism. Perhaps best known for the Obey campaign (Andre the Giant has Posse) but also his images directly challenging capitalism. Fairey is a contradictory character–using others copyrighted images but threatening to sue those who use his images. And so maybe creating the icon images for Obama, who is little distanced from the corporate, capitalist interests Fairey’s work critiques, are just part of that contradiction. Fairey himself says he is too corporate for the street artists and too street for the establishment.

But hope and change in education are not what the Obama posse is delivering. In fact, it is intensification of what came before–even higher (and probably national) standards, ‘better’ (and probably national) tests, support for charter schools, and now teacher pay for test results are what is being offered. The initial enthusiasm of new resources for education from the stimulus package is fading in the face of draconian demands from Education Secretary Arne Duncan and it becomes clearer with each speech that Obama is misinformed about the facts and this misinformation is critical to perpetuating the sense of crisis in education. Obama claimed that school drop out rates have increased three-fold when drop out rates have dropped by a third; he claimed that 8th grade achievement has dropped, when in fact it has risen; and set up fake goals–like having the highest proportion of college graduates in the world, when the US is almost there already.

It would have given me hope for change had Obama called for an evaluation of NCLB, even in small ways such as an analysis of NAEP scores pre NCLB and now. It would have given me hope for change if Obama demonstrated some understanding of the inevitable corruptability of high stakes assessment, whether in the form of mc tests or performance tasks. It would have given me hope for change had Obama focused on ways to cultivate options within public schools and promoted magnet schools rather than charter schools.

I don’t recall who said, authority has no wisdom, but methinks we are seeing this is action.

Serving the Public Interest through Educational Evaluation

This is a pre-publication version of a chapter that analyzes the nature of educational evaluation in a global, neo-liberalist world. The chapter includes some advice to evaluators on how to take back evaluation to serve democratic values, that is, the inclusion of all stakeholders (especially those most often shut out) and open deliberation about what it means for education and schooling to be good or bad.

This chapter will appear in Ryan & Cousins’ edited International Handbook of Educational Evaluation to be published by Sage.

Obama’s “new culture of accountability in America’s schools”

There is no reason to be surprised that Obama continues with his teacher pay for results mantra, and he now adds the expansion of charter schools. There is much babble about “data driven” decisions, but as always the devil is in the data. Apparently Obama hasn’t bothered to look very closely at the research on charter schools, which in general should not give any policy maker comfort in giving the nod to the expansion of these publicly funded ‘private’ schools. Overall, student test scores are lower in charter schools. And, where are the data that point clearly to a connection between forms of teacher compensation and improvements in student learning?

I’m not sure what the NEW culture of accountability is meant to be, but I am guessing that this new culture has all the potential to be scarier and more destructive than the old culture. The Clinton and Bush administrations have buried American education deep under neoliberal regulatory accountability. The NEW culture seems to be about more not different, with even more regulatory requirements driven by the “yes, you can” experiences Obama identifies as the sources of his own success (those fabulous early morning tutoring sessions with his mother).

There is nothing to give one optimism here, and whatever good Obama might do for the country is not going to manifest itself in improvements to education and schooling under these plans.

Here are links to a couple of Jerry Bracey pieces in the Huffington Post that point to the hypocrisy and sophistry in Obama’s education plan.

Bracey #1

Bracey #2

Public Good and Private Interest in Educational Evaluation

Cite as:
Mathison, S. (2009). Public Good and Private Interest in Educational Evaluation. In W. Ayers (Ed.). Handbook of social justice in education. New York: Routledge, pp. 5-14.