Category Archives: assessment

alternatives to standardized testing

67547_10151545229449501_624032542_nIn educational evaluation the global educational reform movement (GERM) has privileged common indicators of student learning outcomes (used in turn for other evaluation purposes like teacher evaluation, even if not a sound practice). There are many reasons why standardized tests become the norm and are reified as the only fair and legitimate way to know how students and schools are doing. There is plenty of literature that debunks that idea.

However, the narrative of standardized testing as a necessary and legitimate means of judging the quality of learning and schooling is powerful and political. In a short commentary for NPR a reporter, Anya Kamenetz, nicely summarizes reasonable alternatives, and these are excellent talking points when confronted with the question, “If not standardized tests, what then?” You can read the article, but in summary:

1) use some sort of matrix sampling (a good idea from NAEP)

2) consider ongoing embedded assessments (this is usually computer based testing)

3) think about what you want to know and it will require multiple measures (in other words, knowing scores in a few subject areas will never be enough, and maybe there are things worth knowing beyond the obvious)

4) start considering novel approaches to assessment, like game based assessment and the not so novel use of portfolios or narrative evaluations

5) think differently about what it means to judge a school and that means looking at more than just narrow student outcomes (school inspections are a notion worth revisiting).

Reporting evaluation results ~ the case of school report cards

Report cards are a particular example of reporting evaluation results ~ telling students and their parents how well they are doing at what they are supposed to be learning. Typically, report cards are done three times in a school year and this time of year is about when that first report card comes home. I did a Q & A about report cards, and had this to say:

A UBC education expert explains why rewarding good grades is an automatic fail

Whether it’s straight As or straight Cs, UBC education professor Sandra Mathison says parents should ask plenty of questions about a student’s report card, and cautions against rewarding good grades.

How useful are report cards?

Prof. Sandra Mathison

There are different perceptions about what types of report cards are useful. What educators see as being appropriate is often not a view shared by parents. This is a fundamental contradiction.

Parents see marks within a framework of competition. Knowing that your child is getting an A means they’re in the top group, and if they get a C, then your child is doing a lot worse than some classmates.

For educators, the evolution of report cards is about providing information that is substantive and communicates, more specifically, the knowledge and skills that students are acquiring and demonstrating. That requires more detail. For example, educators can break “reading skills” down into phonemic awareness, comprehension, vocabulary, and so on.

Having detailed information communicates more clearly what kids know, and the move in most report cards is towards that. But parents often push back and say, “That’s too much information. I just want to know if my kid is doing OK.”

Are letter grades helpful?

Most of the time letter grades represent big categories and are therefore devoid of much meaning. If a high-school student gets a B in chemistry, what does that mean? Chemistry is complex. It has many components. Does the student understand the concepts, but struggles with those ideas in an applied laboratory context? Does the student understand the ideas but has a difficult time with nomenclature?

How should parents react to a less than stellar report card?

It’s more appropriate to ask what sense parents should make of report cards in general. A parent whose child gets all As should be asking questions, just like a parent whose child gets all Cs or Ds or Fs. Simply accepting some general level of performance, whether it’s very good or not, suggests you don’t actually want to know anything detailed about what your child is learning.

All parents, regardless of the letter grade, should say, “Help me understand in more detail what my child knows and can do, and what my child doesn’t seem to know and is having trouble doing.”

Ask the teacher to be analytic about the child’s knowledge and performance, and to show the parent work the student has done that shows something they’re struggling with. Then the parent can see and ask, “What can you and I do in order to help this child?”

Is it helpful to reward good grades with money or gifts?

It’s highly problematic. From an educational research perspective, we foster kids learning things because they have intrinsic value, not because they have rewards attached.

Parents and teachers should foster children’s interest in learning for the sake of learning, not the pursuit of grades or external rewards. When you give cash or gifts to students, you’re saying to them that what matters most are the outcomes. And when you put the emphasis on simple outcomes, such as getting an A or a particular score on a standardized test, it becomes a defining feature of learning. The quality of student thinking and their interest in the content is diminished when they focus on getting grades for gifts.

This short piece generated more media attention than I anticipated, perhaps because many of those in the media are themselves parents facing their children’s report cards.

A follow-up to this short piece was my segment on Stephen Quinn’s CBC On the Coast. Listen here.

Should we give grades in school?

A few school districts in western Canada have moved away from percentage grades to categorical grades and involving students and parents genuinely in conferences about learning. In BC, Maple Ridge and Pitt Meadows school district has replaced letter grades with what they are calling a student-inclusive conferencing model. Battle Creek school district in Alberta has replaced percentage grades with a categorical grading of beginning, developing, achieving, or excelling. This change was implemented some time ago for elementary and junior high schools, and is now being extended to the high school. In both cases, participating in the new grading systems is optional for teachers. The change, in both cases, has been controversial… yay-sayers and nay-sayers abound. In AB there have been parent and student protests.

Today, I was on CBC Radio 1, The 180 with Jim Brown, debating the use of grades in school ~ Michael Zwaagstra, who is affiliated with the neo-liberal Frontier Center for Public Policy, representing the “we can’t live without percentage grades position” and I representing the “schools would be better places without grades position.”

CBC’s mini online poll shows about equal numbers of voters to be for percentage grades and for other approaches to grading.

Click here to hear the show (the interview/debate happens in the second half hour).

Holding accountability to account

One of the hallmarks of any quality evaluation is that it ought to be subject itself to evaluation. Many evaluation schemes in education, such as the test driven accountability scheme, are not evaluated. illusion_of_success_EN The Action Canada Task Force on Standardized Testing has released a report analyzing the place of standardized testing as an accountability measure in Canadian K-12 education systems, using Ontario as a case study focus. “A review of standardized testing in this province and others is not only timely – it’s urgently needed,” says Sébastien Després, a 2012-2013 Action Canada Fellow and co-author of the report.

The Task Force offers four recommendations that could be the heart of an evaluation of accountability schemes in K-12 education across Canada.

Recommendations
We recommend that the Ontario government establish a suitable panel with a balanced and diverse set of experts to conduct a follow-up review of its standardized testing program. In particular:

A. Structure of the tests relative to objectives
i. The panel should review whether the scope of the current testing system continues to facilitate achievement of education system objectives.
ii. The panel should review whether the scale and frequency of testing remains consistent with the Ministry of Education’s objectives for EQAO testing.

B. Impact of testing within the classroom
i. The panel should review the impact on learning that results from classroom time devoted to test preparation and administration.
ii. The panel should review the impact of testing methods and instruments on broader skills and knowledge acquisition.
iii. The panel should review the appropriateness and impact of the pressure exerted by standardized testing on teachers and students.

C. Validity of test results
i. The panel should review whether or not standardized testing provides an assurance that students are performing according to the standards set for them.
ii. The panel should review the impact of measuring progress by taking a limited number of samples throughout a student’s career.

D. Public reporting and use of test results
i. The panel should review the impact of the potential misinterpretation and misuse of testing results data, and methods for ensuring they are used as intended.
ii. The panel should review supplemental or alternative methods of achieving public accountability of the educational system.

Evaluating schools

If nothing else, we have learned a great deal about what doesn’t work in terms of evaluating schools. The global penchant for using a few outcomes measures just doesn’t do the trick… this is perhaps most obvious in the USA where judging the quality of schools continues to spiral downward from NCLB to Race to the Top, but around the world we see a similar story. And, we see a few counterpoints, such as the success of the Finnish school system where the focus is decidedly not on standardized outcomes on a few measures.

In British Columbia, Canada where education is a decided provincial matter and where provincial politics can actually lead to quite radical shifts in policies and programmatic initiatives, this is a moment of potential change. BC schools have been for many years now held hostage by the scores on the Foundation Skills Assessment (FSA), a test given to all 4th and 7th grade students in the province. Support for the FSA has been eroding over the past several years with a chorus of skeptical teacher, school administrator and school trustee voices.

One initiative, The Great Schools Project, has been developing alternative ideas about school evaluation. The website gives s sense of the GSP platform and a bit more information about the issues can be heard in a segment of a local radio talk show.

Obama’s “new culture of accountability in America’s schools”

There is no reason to be surprised that Obama continues with his teacher pay for results mantra, and he now adds the expansion of charter schools. There is much babble about “data driven” decisions, but as always the devil is in the data. Apparently Obama hasn’t bothered to look very closely at the research on charter schools, which in general should not give any policy maker comfort in giving the nod to the expansion of these publicly funded ‘private’ schools. Overall, student test scores are lower in charter schools. And, where are the data that point clearly to a connection between forms of teacher compensation and improvements in student learning?

I’m not sure what the NEW culture of accountability is meant to be, but I am guessing that this new culture has all the potential to be scarier and more destructive than the old culture. The Clinton and Bush administrations have buried American education deep under neoliberal regulatory accountability. The NEW culture seems to be about more not different, with even more regulatory requirements driven by the “yes, you can” experiences Obama identifies as the sources of his own success (those fabulous early morning tutoring sessions with his mother).

There is nothing to give one optimism here, and whatever good Obama might do for the country is not going to manifest itself in improvements to education and schooling under these plans.

Here are links to a couple of Jerry Bracey pieces in the Huffington Post that point to the hypocrisy and sophistry in Obama’s education plan.

Bracey #1

Bracey #2