Category Archives: Educational evaluation

Thinking about the cost of evaluation

Any evaluation must consider whether the indicators to be used are truly available and high quality, and whether the cost of data collection are warranted. The failure of the four large testing companies to satisfactorily meet state testing demands created by NCLB is an excellent example of the triumph of ideology over reasoned evaluation practice. The NCLB requirement that all students in grades 3 trhough 8 be tested has created a boom for the testing industry. However, the oligopoly of the four major testing companies cannot meet the demand nor do the job well. The incidence of errors is widespread and the inability of these companies to deliver the test scores accurately and in a timely manner is apparent across the US. The article US Testing Companies Buckling Under Weight of NCLB illustrates the pervasiveness of the problems. What the article does not mention is the deep and long standing connections between the Bush family and the testing industry, particularly CTB/McGraw Hill.

On the ranking of schools in BC

It is nice to see the Georgia Straight reporting on schooling issues with this week’s story on the debate between Paul Shaker (Dean of Education at SFU) and Peter Crowley of the Fraser Institute.

Shaker challenges the overuse and over interpretation of FSA scores as indicators of the quality of schools. Although lots of numbers may look scientific, Shaker asserts the Fraser Institute approach would never stand up under serious peer review.

Read the Georgia Straight article here.

If only school was this interesting…

9daydreaming.jpg
These are not kids in a classroom preparing to take whatever state mandated tests in whatever state. The world Maira Kalman creates in her illustrations and poetry is funny, sad, stunningly prophetic, and whimsical. This classroom reflects a decidedly pre-NCLB utopia, that never existed, but perhaps dwells in our minds sufficiently to remember there is an alternative.

Note: Maira Kalman’s books have been read at my house for years, by adults and children alike. She has lightened the instructions for good English in an illustrated version of Strunk and White

It is amazing what misuses educational tests are put to…

Educational and psychological tests have pockets of darkness in their history. A key one is eugenics, the continued connections between mental measurement and racism. The unbelievably goofy pseudo-scientist J. Phillipe Rushton is featured in my home town newspaper, the Vancouver Sun, for his silly study in which he asserts scores on the SAT demonstrate men are naturally smarter than women. Apparently no one told him the SAT doesn’t even predict success in college, let alone success in life. Apparently he is also unaware that even the test publisher says the test isn’t worth much, especially in judgements of merit, when a point spread of at least 125 is necessary to even suggest real differences exist. Rushton can’t even keep his silly science straight – he previously claimed there is an inverse relationship between intelligence and penis size. Seems to me the women have it all over the men based on this theory.

The never-ending story of grade inflation

Once again the Chronicle of Higher Education covers the story everyone already knows–grade inflation has gone to college. Read When B’s are Better if you are the only person not familiar with this topic. The nostalgia for the days of higher standards and fewer A’s misses most of what is important about the relationship between grades and what students know and can do. It is, of course, entirely possible for everyone to get an A (which is not to say that is currently the case) but that requires a criterion based approach to assessment. Most arguments about grade inflation simply invoke the normal curve as the natural distribution of knowledge. Rather than asking professors to give fewer A’s it would be more interesting to ask professors to clarify what they think the A they are giving means. In other words, specify the meaning of the grade, not necessarily change it. And, all of this talk about grade inflation dodges the complexity of the higher education context where grades are fetishes for students and become the currency that allows faculty and students to get what they want within a system not necessarily friendly to either.

Student Experiences of High Stakes Testing

Hannah before, during, after.jpgThe concluding component of a five year NSF funded research project exploring the impacts of state mandated testing in elementary and middle schools in three upstate NY districts is a small study of kid’s perceptions of the experience. Over a year we meet with a small group of 4th and 8th graders–they provided an array of data including journalling, drawing and interviews. Many of the images and an analysis of some of these images is posted on my website.