e-valuation

This is a pre-publication version of a chapter that analyzes the nature of educational evaluation in a global, neo-liberalist world. The chapter includes some advice to evaluators on how to take back evaluation to serve democratic values, that is, the inclusion of all stakeholders (especially those most often shut out) and open deliberation about what it means for education and schooling to be good or bad.

This chapter will appear in Ryan & Cousins’ edited International Handbook of Educational Evaluation to be published by Sage.

There is no reason to be surprised that Obama continues with his teacher pay for results mantra, and he now adds the expansion of charter schools. There is much babble about “data driven” decisions, but as always the devil is in the data. Apparently Obama hasn’t bothered to look very closely at the research on charter schools, which in general should not give any policy maker comfort in giving the nod to the expansion of these publicly funded ‘private’ schools. Overall, student test scores are lower in charter schools. And, where are the data that point clearly to a connection between forms of teacher compensation and improvements in student learning?

I’m not sure what the NEW culture of accountability is meant to be, but I am guessing that this new culture has all the potential to be scarier and more destructive than the old culture. The Clinton and Bush administrations have buried American education deep under neoliberal regulatory accountability. The NEW culture seems to be about more not different, with even more regulatory requirements driven by the “yes, you can” experiences Obama identifies as the sources of his own success (those fabulous early morning tutoring sessions with his mother).

There is nothing to give one optimism here, and whatever good Obama might do for the country is not going to manifest itself in improvements to education and schooling under these plans.

Here are links to a couple of Jerry Bracey pieces in the Huffington Post that point to the hypocrisy and sophistry in Obama’s education plan.

Bracey #1

Bracey #2

Public Good and Private Interest in Educational Evaluation

Cite as:
Mathison, S. (2009). Public Good and Private Interest in Educational Evaluation. In W. Ayers (Ed.). Handbook of social justice in education. New York: Routledge, pp. 5-14.

Any evaluation must consider whether the indicators to be used are truly available and high quality, and whether the cost of data collection are warranted. The failure of the four large testing companies to satisfactorily meet state testing demands created by NCLB is an excellent example of the triumph of ideology over reasoned evaluation practice. The NCLB requirement that all students in grades 3 trhough 8 be tested has created a boom for the testing industry. However, the oligopoly of the four major testing companies cannot meet the demand nor do the job well. The incidence of errors is widespread and the inability of these companies to deliver the test scores accurately and in a timely manner is apparent across the US. The article US Testing Companies Buckling Under Weight of NCLB illustrates the pervasiveness of the problems. What the article does not mention is the deep and long standing connections between the Bush family and the testing industry, particularly CTB/McGraw Hill.

Although the mothership NEA is supportive of NCLB and standards based reform approaches, local chapters of the NEA are key to providing alternate perspectives on the impact of the testing regime prescribed by NCLB. A group, which includes the CA NEA, has created the Eliminate NCLB organization. Web-based activism at work.

It is nice to see the Georgia Straight reporting on schooling issues with this week’s story on the debate between Paul Shaker (Dean of Education at SFU) and Peter Crowley of the Fraser Institute.

Shaker challenges the overuse and over interpretation of FSA scores as indicators of the quality of schools. Although lots of numbers may look scientific, Shaker asserts the Fraser Institute approach would never stand up under serious peer review.

Read the Georgia Straight article here.

These are not kids in a classroom preparing to take whatever state mandated tests in whatever state. The world Maira Kalman creates in her illustrations and poetry is funny, sad, stunningly prophetic, and whimsical. This classroom reflects a decidedly pre-NCLB utopia, that never existed, but perhaps dwells in our minds sufficiently to remember there is an alternative.

Note: Maira Kalman’s books have been read at my house for years, by adults and children alike. She has lightened the instructions for good English in an illustrated version of Strunk and White

In a short newspaper article, a school counsellor in Daytona Beach gets right to the heart of so much that is problematic with NCLB.

Read on.

Educational and psychological tests have pockets of darkness in their history. A key one is eugenics, the continued connections between mental measurement and racism. The unbelievably goofy pseudo-scientist J. Phillipe Rushton is featured in my home town newspaper, the Vancouver Sun, for his silly study in which he asserts scores on the SAT demonstrate men are naturally smarter than women. Apparently no one told him the SAT doesn’t even predict success in college, let alone success in life. Apparently he is also unaware that even the test publisher says the test isn’t worth much, especially in judgements of merit, when a point spread of at least 125 is necessary to even suggest real differences exist. Rushton can’t even keep his silly science straight – he previously claimed there is an inverse relationship between intelligence and penis size. Seems to me the women have it all over the men based on this theory.

Once again the Chronicle of Higher Education covers the story everyone already knows–grade inflation has gone to college. Read When B’s are Better if you are the only person not familiar with this topic. The nostalgia for the days of higher standards and fewer A’s misses most of what is important about the relationship between grades and what students know and can do. It is, of course, entirely possible for everyone to get an A (which is not to say that is currently the case) but that requires a criterion based approach to assessment. Most arguments about grade inflation simply invoke the normal curve as the natural distribution of knowledge. Rather than asking professors to give fewer A’s it would be more interesting to ask professors to clarify what they think the A they are giving means. In other words, specify the meaning of the grade, not necessarily change it. And, all of this talk about grade inflation dodges the complexity of the higher education context where grades are fetishes for students and become the currency that allows faculty and students to get what they want within a system not necessarily friendly to either.