Apr 23 2013
Archive for the 'Educational evaluation' Category
Apr 19 2013
A few school districts in western Canada have moved away from percentage grades to categorical grades and involving students and parents genuinely in conferences about learning. In BC, Maple Ridge and Pitt Meadows school district has replaced letter grades with what they are calling a student-inclusive conferencing model. Battle Creek school district in Alberta has replaced percentage grades with a categorical grading of beginning, developing, achieving, or excelling. This change was implemented some time ago for elementary and junior high schools, and is now being extended to the high school. In both cases, participating in the new grading systems is optional for teachers. The change, in both cases, has been controversial… yay-sayers and nay-sayers abound. In AB there have been parent and student protests.
Today, I was on CBC Radio 1, The 180 with Jim Brown, debating the use of grades in school ~ Michael Zwaagstra, who is affiliated with the neo-liberal Frontier Center for Public Policy, representing the “we can’t live without percentage grades position” and I representing the “schools would be better places without grades position.”
Click here to hear the show (the interview/debate happens in the second half hour).
Feb 25 2013
One of the hallmarks of any quality evaluation is that it ought to be subject itself to evaluation. Many evaluation schemes in education, such as the test driven accountability scheme, are not evaluated. The Action Canada Task Force on Standardized Testing has released a report analyzing the place of standardized testing as an accountability measure in Canadian K-12 education systems, using Ontario as a case study focus. “A review of standardized testing in this province and others is not only timely – it’s urgently needed,” says Sébastien Després, a 2012-2013 Action Canada Fellow and co-author of the report.
The Task Force offers four recommendations that could be the heart of an evaluation of accountability schemes in K-12 education across Canada.
We recommend that the Ontario government establish a suitable panel with a balanced and diverse set of experts to conduct a follow-up review of its standardized testing program. In particular:
A. Structure of the tests relative to objectives
i. The panel should review whether the scope of the current testing system continues to facilitate achievement of education system objectives.
ii. The panel should review whether the scale and frequency of testing remains consistent with the Ministry of Education’s objectives for EQAO testing.
B. Impact of testing within the classroom
i. The panel should review the impact on learning that results from classroom time devoted to test preparation and administration.
ii. The panel should review the impact of testing methods and instruments on broader skills and knowledge acquisition.
iii. The panel should review the appropriateness and impact of the pressure exerted by standardized testing on teachers and students.
C. Validity of test results
i. The panel should review whether or not standardized testing provides an assurance that students are performing according to the standards set for them.
ii. The panel should review the impact of measuring progress by taking a limited number of samples throughout a student’s career.
D. Public reporting and use of test results
i. The panel should review the impact of the potential misinterpretation and misuse of testing results data, and methods for ensuring they are used as intended.
ii. The panel should review supplemental or alternative methods of achieving public accountability of the educational system.
Nov 28 2012
If nothing else, we have learned a great deal about what doesn’t work in terms of evaluating schools. The global penchant for using a few outcomes measures just doesn’t do the trick… this is perhaps most obvious in the USA where judging the quality of schools continues to spiral downward from NCLB to Race to the Top, but around the world we see a similar story. And, we see a few counterpoints, such as the success of the Finnish school system where the focus is decidedly not on standardized outcomes on a few measures.
In British Columbia, Canada where education is a decided provincial matter and where provincial politics can actually lead to quite radical shifts in policies and programmatic initiatives, this is a moment of potential change. BC schools have been for many years now held hostage by the scores on the Foundation Skills Assessment (FSA), a test given to all 4th and 7th grade students in the province. Support for the FSA has been eroding over the past several years with a chorus of skeptical teacher, school administrator and school trustee voices.
One initiative, The Great Schools Project, has been developing alternative ideas about school evaluation. The website gives s sense of the GSP platform and a bit more information about the issues can be heard in a segment of a local radio talk show.
Oct 21 2012
While the strategy of students evaluating professors is common in higher education, this approach is rare in K-12 education. One component of the Measures of Effective Teaching Project at Harvard is just such data. Based on a a decade old survey developed by Ronald Ferguson, an economist at Harvard, a shorter survey had been developed that asks students to describe their classroom instructional climate. Importantly, students (all the way from Kindergarten through high school) are not asked to judge their teachers, but to provide a description of what the classroom environment looks and feels like to them. The survey includes the following kinds of questions:
Caring about students (Encouragement and Support)
o Example: The teacher in this class encourages me to do my best.”
• Captivating students (Learning Seems Interesting and Relevant)
o Example: “This class keeps my attention – I don’t get bored.”
• Conferring with students (Students Sense their Ideas are Respected)
o Example: “My teacher gives us time to explain our ideas.”
• Controlling behavior (Culture of Cooperation and Peer Support)
o Example: “Our class stays busy and doesn’t waste time.”
• Clarifying lessons (Success Seems Feasible)
o Example: “When I am confused, my teacher knows how to help me understand.”
• Challenging students (Press for Effort, Perseverance and Rigor)
o Example: “My teacher wants us to use our thinking skills, not just memorize things.”
• Consolidating knowledge (Ideas get Connected and Integrated)
o Example: “My teacher takes the time to summarize what we learn each day.”
A recent story in The Atlantic the results are summarized.
The most refreshing aspect of Ferguson’s survey might be that the results don’t change dramatically depending on students’ race or income… But overall, even in very diverse classes, kids tend to agree about what they see happening day after day.
Whether these data should be used in teacher evaluation requires careful consideration, but from a larger evaluative perspective what this demonstrates is the very valuable data that those who are meant to benefit most from programs and interventions can provide. If you ask the right questions, and if you respect their experiences and perspectives.
Sep 24 2012
This is a pre-publication version of an entry in the International Encyclopedia of Education, 3rd Edition. Please note the correct citation in the text and refer to the final version in the print version of the IEE.
Mathison, S. (2010). The purpose of evaluation. In P. Peterson, B. McGaw & E. Baker (Eds.). The International Encyclopedia of Education, 3rd ed. Elsevier Publishers.
There are two primary purposes of evaluation in education: accountability and amelioration. Both purposes operate at multiple levels in education from individual learning to bounded, focused interventions to whole organizations, such as schools or colleges. Accountability is based primarily on summative evaluations, that is, evaluations of fully formed evaluands and are often used for making selection and resource allocation decisions. Amelioration is based primarily on formative evaluation, that is, evaluations of plans or developing evaluands and are used to facilitate planning and improvement. Socio-political forces influence the purpose of evaluation.
Sep 06 2012
NEPC brief on parent trigger laws: MISSING THE TARGET? THE PARENT TRIGGER AS A STRATEGY FOR PARENTAL ENGAGEMENT AND SCHOOL REFORM
With the impending release of the movie Won’t Back Down, NEPC authors provide a critique of parent trigger laws.
There is so much that is wrong headed about these laws and the NEPC brief does a nice job of touching on the major points: the misconstrual of parent involvement as the primary key to school reform; the representation of the laws as a grassroots movement when it is funded and promoted by think tanks (DfER, Parent Revolution, and StudentsFirst) and foundations (Gates, Broad, Walton Foundations) who are often right wing; and conflates parent control of schools with the promotion of charter schools as the only alternative to public schooling.
Jun 14 2012
The full story is in the Teacher, the BCTF monthly news magazine.
Jun 11 2012
The emphasis on assessing student learning through large scale standardized and high stakes testing programs has not been an effective strategy for school reform and improvement. No real surprise there. Reform has turned on teacher performance, often guaged by the very tests that are so fraught with socio-political and technical problems. Too often moving onto another strategy is an ideological move with precious little evidence one way or the other. While there are problems with this study that looks at how teacher evaluation is conceptualized and conducted in these 10 places, the effort to engage in evaluation of the evaluations is laudable.