e-valuation

There can be no doubt that technology can make the work of evaluation easier and the array of software and applications is evergrowing. Check out the AEA365 blog and search for technology related posts… there are plenty. The challenge for evaluators will be thoughtful use of technology and avoiding technology driven evaluation practices. One of the best examples of technology driving and structuring thinking, knowledge construction and presentation is Powerpoint… Microsoft has created simple software that too often control what counts as information. Edward Tufte’s critique of ppt, PowerPoint Does Rocket Science–and Better Techniques for Technical Reports, is required reading for anyone who has ever or will ever use powerpoint.

Today Apple revealed the availability of, Look For, an iphone app for recording classroom observations of teaching and it is marketed as a tool for teacher evaluation. With a quick click (and some added notes if you like) principals can record whether teachers are “making subject matter meaningful” or “facilitating the learning process.” The promo for Look For says the app has the following features:
-Create unlimited observations
-Sort observations by school, teacher, subject and date
-Select from hundreds of qualification points within 6 basic categories
-Easily email and share reports and progress instantly
-Track teacher progress through each of the 6 instructional categories
-Supports state and national standards

Everyone wants technology to make their lives and jobs easier, and principals are no exception. But is this like ppt? An app that pre-defines and standardizes what counts as good teaching and limits sensitivity to context may be time saving, but does it promote good evaluation? Establishing criteria is key to good evaluation, but this is and ought to be a slippery part of the process… we cannot and should not know all of the relevant criteria a priori and we ought to be open to recognizing good and bad making attributes of teaching in situ. Principals and teachers need to be able to recognize and acknowledge what is not easily or necessarily captured by the 6 instructional categories.

So maybe Look For is a good app, but only if used in a critical way… true for all technology.

If you doubt that neo-liberalism dominates the educational reform landscape take a look at this NYT story, which does a nice job of following the Gates Foundation money and how the spending has influenced the adoption of national curriculum standards, worked against teacher unions, and infiltrated school districts, think tanks and even the unions. This strategy is not new and is exactly what right wing Christian groups have done to influence schools by getting elected/appointed to school board positions. In both cases, there is something unsavory about the subterfuge, the lack of transparency, the buying of influence… as opposed to public deliberations about the schools we want and how to get them.

Just as it makes little sense to talk about the validity of a test, it makes little sense to talk about a formative test. Although there is a good emphasis on formative assessment of student learning, there is an unfortunate confusion about what the formative means. Too often the instrument is identified as formative, when in fact it is how the information from the instrument is used that makes the evaluation formative. The same test and the results of that test can be used either formatively or summatively. Just as the test is not valid (it is the inferences that are made that have or lack validity), neither is the test itself formative or summative. Popham has a nice little discussion of this in his Ed Week piece Formative Assessment–A Process, Not a Test.

Good report on the flaw in some current thinking about teacher evaluation from NEPC. The value added approach is a simplistic strategy for determining teaching effectiveness.

Due Diligence and the Evaluation of Teachers
by Derek C. Briggs, Ben Domingue
February 8, 2011

The research on which the Los Angeles Times relied for its August 2010 teacher effectiveness reporting was demonstrably inadequate to support the published rankings. Using the same L.A. Unified School District data and the same methods as the Times, this study probes deeper and finds the earlier research to have serious weaknesses.

Gotta love Alfie Kohn, and in this Huffington Post article he clearly illustrates the problems with the value added (meaning increased test scores) approach to evaluating teacher performance. But Kohn’s article also points out common misconceptions in many evaluation contexts, like uniformity is the same as quality and value is easily perverted or narrowed by the indicators selected.

A new blog from the AFT, What Should Count, that is self described as follows:

The American Federation of Teachers believes that accountability should be about making sure students have resources to learn and succeed: rich curricula, excellent facilities, talented—and well-supported—faculty, and robust academic standards that are devised and improved by the people who deliver them. This website is designed to serve not only as a clearinghouse of accountability initiatives at the international, national, state and local levels, but also as a starting point for discussing accountability systems that best help our students succeed.

Time will tell whether the AFT contributes positively to the discourse on assessment K-16, but they do have some atoning to do, so this may be a positive start. With Albert Shanker as president, the AFT embraced the standards and assessment reform that began with A Nation at Risk and supported testing new teachers. Initially the AFT neither endorsed or opposed NCLB, but Sandra Feldman’s 2003 comments suggest an endorsement:

The federal NCLB Act poses yet another test of our ability to be con- structive, responsive, and creative while simultaneously fighting and protecting against the indefensible. The law is built around goals we’ve long supported: high academic standards and achievement, eradicating achievement gaps between the haves and the have-nots, making sure that every teacher in every school is qualified, and, yes, accountability. The law also mandates reporting outcomes by student subgroup which is the right thing to do because it puts inequities out there for all to see. (Feldman, S. (2003, July). Keynote address to the AFT QuEST (Quality Education Standards in Teaching) conference, Washington, DC.)

AFT’s opposition to test driven reform has, however, been sharper in recent years in response to pressure from its rank and file members. The AFT’s conservatism and strategy of working behind the scenes doesn’t obviously position the teacher union that represents most urban school teachers as a force for change. Current president Randi Weingarten’s testimony before the House Committee on Education and Labor suggests a continued support for national standards, standardization, and using test scores (at least in part) in determining teacher pay.
So, good for the AFT for creating this blog, but here’s hoping they do much more to contribute to a quality work life for their members, and quality education for children living in US cities.

While military academies have been part of Chicago schools since 1999, the surge in efforts to create more is palpable, and Arne Duncan is a big supporter (no surprise there). The latest development is the financial incentive to local often financially strapped school districts to open charter schools run by the Marine Corps. Little to no evidence suggests that these schools improve the quality of education or academic achievement for students. However, Congress has passed a bill that supports increases in Junior ROTC. In the absence of evidence that such military academies are better schools, one has to conclude that they serve other purposes–fueling the military ranks and using vulnerable students for the benefit of the corporate-military apparatus.

Evaluations of all charter schools, including military academies need to consider a broad context and consider their worth and value by taking into consideration more than the simplistic representations of their value and dig through a state supported rhetoric that is at best diversionary.

http://www.msnbc.msn.com/id/31108509/

Michele Lamont in How Professors Think: Inside the Curious World of Academic Judgment opens the Pandora’s Box of peer review, the primary form of evaluation in higher education. Lamont’s curiosity, like Pandora’s, reveals secretive deliberations that all too often amount to judgments of quality based on the similarity of the work being judged to that of the judges. Lamont examines differences across disciplines, highlights the tension between the idea of having independently established criteria and standards and the inevitability of situational deliberation on what is good or bad, and ultimately calls for a more open, transparent approach to evaluation in higher education. In this later move, she searches for the hope that Pandora found at the bottom of the box.

Lamont describes the details of her book in a short essay in the Huffington Post.

I have to confess I was disappointed when Shepard Fairey developed the posters for Obama’s campaign. Shepard Fairey, before the Obama campaign, a skateboarder and graffiti artist stood for a challenge to authority, war and capitalism. Perhaps best known for the Obey campaign (Andre the Giant has  Posse) but also his images directly challenging capitalism. Fairey is a contradictory character–using others copyrighted images but threatening to sue those who use his images. And so maybe creating the icon images for Obama, who is little distanced from the corporate, capitalist interests Fairey’s work critiques, are just part of that contradiction. Fairey himself says he is too corporate for the street artists and too street for the establishment.

But hope and change in education are not what the Obama posse is delivering. In fact, it is intensification of what came before–even higher (and probably national) standards, ‘better’ (and probably national) tests, support for charter schools, and now teacher pay for test results are what is being offered. The initial enthusiasm of new resources for education from the stimulus package is fading in the face of draconian demands from Education Secretary Arne Duncan and it becomes clearer with each speech that Obama is misinformed about the facts and this misinformation is critical to perpetuating the sense of crisis in education. Obama claimed that school drop out rates have increased three-fold when drop out rates have dropped by a third; he claimed that 8th grade achievement has dropped, when in fact it has risen; and set up fake goals–like having the highest proportion of college graduates in the world, when the US is almost there already.

It would have given me hope for change had Obama called for an evaluation of NCLB, even in small ways such as an analysis of NAEP scores pre NCLB and now. It would have given me hope for change if Obama demonstrated some understanding of the inevitable corruptability of high stakes assessment, whether in the form of mc tests or performance tasks. It would have given me hope for change had Obama focused on ways to cultivate options within public schools and promoted magnet schools rather than charter schools.

I don’t recall who said, authority has no wisdom, but methinks we are seeing this is action.