Category Archives: Curriculum Evaluation

the difference between external and independent evaluation

The terms external and independent evaluation are often conflated, largely because external evaluations are (wrongly) assumed to be more independent than internal evaluations. A good example is the evaluation of the LAUSD iPad initiative conducted by the American Institutes for Research, which is described in an EdWeek story like this:

An independent evaluation of the Los Angeles Unified School District’s ambitious—and much-maligned—effort to provide digital devices to all students found that the new, multi-million dollar digital curriculum purchased as part of the initiative was seldom used last year because it had gaping holes, was seen by some teachers to lack rigor, and was plagued by technical glitches.

To be fair, AIR calls their evaluation external, NOT independent. And the evaluation IS external because the evaluators (AIR staff) are not members of the organization (LAUSD) in which the evaluand exists. They are external also to the iPad initiative, the program itself.

Screen Shot 2014-09-19 at 10.15.22 AMWhile a bit pedestrian, it is worth asking what is meant by independent so it is clearer how external and independent are not synonyms.

Using the LAUSD iPad example, is AIR’s evaluation independent? The first sense of independence would suggest the evaluation is free from control by any one outside of AIR and the AIR evaluation team ~ that the evaluation is not influenced by any one, including the LAUSD, Pearson or Apple. It is clear from the report that indeed the evaluation is influenced by the LAUSD by asking questions that are relevant and desirable to them, although there is no obvious influence from Pearson or Apple, the two corporations providing the hardware, software, and professional development. This is absolutely typical in evaluation ~ those who commission the evaluation influence the focus of the evaluation, and often how the evaluation is done (although whether that was the case in this evaluation is not explicit in the report).

A key to the influence the LAUSD has on the evaluation is illustrated in the description of the program milestones, the first of which is characterized as awarding the contract to Apple in June 2013. But it is clear this is not the first milestone as a LAUSD Board report released in August 2014 points to Superintendent Deasy’s manipulation of the bidding process so it would be a foregone conclusion the successful vendor would be the Apple/Pearson combo. AIR evaluators would have known about this. There is also no mention of the LAUSD’s refusal, when the project was rolled out, to reveal how much money had been paid to Pearson, a subcontractor to Apple on the $30 million first phase of the project. 

Evaluators might argue that these matters are not the focus of the evaluation as framed by the evaluation questions, and that is likely true. The problem is that the evaluation questions are usually (and no reason to believe this wasn’t the case with the AIR evaluation of the iPad initiative) mutually agreed upon by the external evaluator and the organization contracting for the evaluation. That an organization would not want to include issues of malfeasance, transparency and accountability is understandable in many cases. A truly independent evaluation would necessarily include these issues, as well as other unanticipated circumstances and outcomes. The lack of independence is structural (in who commissions evaluations) privileging the perspectives of decision-makers, funders and CEOs.

The second sense of independence points to a failure for every external evaluation ~ external evaluators are in an immediate sense dependent on whomever commissions the evaluation for their subsistence and in the longer term sense if they wish to do evaluations for this organization again, or even other organizations who may monitor how the first sense of independence is treated in past evaluations. External evaluations lack financial independence.

And, external evaluations fail on the third sense of independence because the evaluators and the organizations commissioning evaluations of themselves or their programs are connected to one another, certainly financially but also often in an ongoing relationship with one another.

Whose interests are served and how?

Screen Shot 2014-09-19 at 11.53.22 AMBecause of the lack of structural and financial independence, external evaluations (as much as internal evaluations) emphasize some interests and serve some ends, while ignoring or bracketing others. In the LAUSD iPad initiative, the interests of both the LAUSD as a whole, the Board, and John Deasy are served both by what is included and excluded. The AIR evaluation provides a good descriptive account of the roll out of a major technology initiative, including issues with levels and types of use, quality of curriculum, and what worked well (the use of apps, for example). The evaluation could not be construed as positive on the Pearson curriculum content.

But by avoiding the inclusion of issues around the initial bidding process, so too are specific interests of Deasy, Apple and Pearson served. What does it mean that both Deasy and Apple were involved in manipulating the bidding for the contract? Put in the context of Apple’s aggressive marketing of iPads to schools, this becomes potentially an example of profit-making over learning. Apple’s last quarterly earnings claims more than 13 million iPads have been sold globally for education; 2 and a half iPads are sold for every Mac in K-12 education. The secretive partnering with Pearson, a company recognized more for making profit than making educational gains, should be an additional piece of an independent evaluation. Corporations whose primary interest is profit making and who mastermind programs and products deserve scrutiny for how their interests intersect with other interests (like teaching and learning).

Although there are few mechanisms for truly independent evaluations, professional evaluation associations and professional evaluators should be pondering how their work as either internal or external evaluators might be more independent, as well as developing strategies for conducting truly independent evaluations that are simply not compromised by the structural and financial relationships that characterize virtually all evaluations.

Curriculum by K. Teitlebaum

Read this chapter on the history and definition of curriculum by Ken Teitlebaum. This chapter should be cited as

Teitlebaum, K. (2008). Curriculum. In S. Mathison & E. W. Ross (eds.). Battleground School. New York: Greenwood Press.

Teitlebaum provides a sketch of the milestones in the history of curriculum.Curriculum
Kenneth Teitelbaum

There is a sense in which the sum total of what is taught in school, which for the moment we can think of as the curriculum, is ignored as a debatable topic as educational theorists and practitioners worry more about how to teach, how to group, how to find and organize materials, how to assess, and how to report results for what is assumed should be taught because it is included in the textbook, the curriculum guide, or district or state directives that were issued just recently or during some long ago period. That is, the critical examination of the actual academic content, skills, dispositions, appreciations, and ways of learning to be planned, designed, taught and assessed in an educational setting is taken for granted, approached as unproblematic, accepted as the conventional and commonsensical wisdom, not the object of thorough and sustained scrutiny. Books and articles that tout “best practices” of curriculum design and organization and instructional and assessment strategies help teachers and administrators to do better what presumably needs to take place in the classroom.

On the other hand, a careful examination of educational discourse and policymaking, starting from the early years of the twentieth century—contained in scholarly books and journals as well as countless articles that appear in popular newspapers and magazines—belies this sense of consensus about and lack of attention to substantive issues of curriculum. Indeed, it is not uncommon for the title of this collection or something very much like it to be referred to in headlines that highlight the contested terrain that is curriculum work, as in a recent article in The Wall Street Journal that announced one content area (sex education) as having become the “latest school battleground.” If “culture wars” exist, as many scholars and media pundits would have us believe, they involve not just political parties, professional organizations, and national and community interest groups, but our classrooms as well.

Put simply, there exists a universe of knowledge, skills, etc. from which to select a relatively small amount to comprise a curriculum; the curriculum can be planned and organized in a variety of ways; deliberations about curriculum choices can take many different forms and involve many different stakeholders; and the results of what we plan for and do in the classroom can be assessed in a range of ways, with regard to both its educational and socio-political ramifications. There continues, always, somewhere to be many choices to make, and there is much contention, even if in partially stifled voices, about whether or not we are resolving curriculum questions—of content selection, for example—in the best, most effective and most equitable ways for/in a democratic society.


Several different areas of controversy regarding curriculum can be identified, starting with its very definition. While arguments about definition can seem stodgy and irrelevant to what takes place in the classroom, in fact the way one conceives of curriculum can have a significant impact on its study and practice. The fact that there are estimated to be more than 120 definitions in the professional literature, taking us well beyond the word’s Latin root of “racecourse,” perhaps testifies to the importance and variance of its basic characteristics.

Thus, for example, James Popham and Eva Baker’s well-known definition in Systematic Instruction stresses “all planned learning outcomes for which the school is responsible.” Such a definition places a priority on what has been planned and what can be determined to be learned. Curriculum developers and evaluators (and scholars) would have such characteristics uppermost in their minds when studying, planning, organizing and assessing the curriculum. But what about what occurs and is observable in classrooms that are not planned, e.g., what are often referred to as “teachable moments”? Do they have no significance with regard to student learning and what can be considered part of the curriculum that students experience? And what about those experiences that are planned (or unplanned) for which learning outcomes are not easily determined, especially in the short run, such as appreciations and more expressive ways of learning? Should they be excluded from consideration and perhaps downplayed with regard to their potential educational value?

Another definition focuses on all experiences children have under the guidance of teachers, which de-emphasizes the planned and at the same time specifies the teacher’s role. Some might argue that the experiences students have in school that are not under the direct guidance of teachers should also be considered (and assessed) as part of the curriculum, for example, what occurs in playgrounds, hallways, cafeterias, and the like. Some curriculum scholars posit curriculum as the interrelated complex set of plans and experiences that students have under the guidance of the school. This definition highlights “plans” but does seem to allow for the unplanned (experiences) as well (though what exactly is meant by “interrelated” is unclear). Moreover, it does not specify the teacher as the only school participant that is involved with what can be considered the curriculum, placing the emphasis instead on what “a student undertakes.” Anyone adopting such a definition, then, will give careful attention to all experiences that students have in school, which will include more than just what the teacher does in the classroom.

Another definition was offered almost 100 years ago by J. Franklin Bobbitt in his book, The Curriculum (1918), which is generally recognized to be the first textbook focused specifically on “curriculum.” For Bobbitt, the curriculum was “that series of things which children and youth must do and experience by way of developing abilities to do the things well that make up the affairs of adult life; and to be in all respects what adults should do” (p. 42; emphasis in original). This definition implies a broadening of the scope beyond what takes place directly in school. At the same time, it specifically fosters the notion that the curriculum should be primarily (or exclusively) about “abilities to do things.” Enhanced understandings and deeply held appreciations, for example, are apparently not included. In addition, preparing for “the affairs of adult life” is primary, rather than, for example, what children are experiencing (or reflecting about) currently in their lives or critically examining social conditions. This definition also appears to assume that there is a consensus about the things that all “children and youth must do and experience,” that is, what the most important “affairs of adult life” actually are and what specifically “adults should do.” All aspects of a definition have serious repercussions not only for one’s view of what is (should be) taught and evaluated in school but also with regard to what researchers might emphasize in their own work in addressing the curriculum. If one accepts Bobbitt’s definition, much time needs to be spent in determining the specific “things” that children and youth must learn to do in school and how they align with what adults presumably should do.

There are many other definitions of curriculum that could be considered. The point is that the curriculum is an applied field of inquiry whose very definition has been a “battleground” of sorts, with significant implications for the way one determines school knowledge as well as approaches to teaching, learning and evaluation. The definition one adopts also sets parameters for the kind of research that is most meaningful, including the extent to which one attends to the hidden curriculum (i.e., institutional norms and values not openly acknowledged by school officials that are in fact immersed in classrooms and schools) and the null curriculum (i.e., subject matter and ways of learning that are not selected to be a part of school life). It can even indicate whether or not the exclusive focus of one’s attention should be on the classroom or school life in general, or, perhaps, broader institutional and discursive practices, structures, images and experiences that go beyond the school setting.
Different conceptions, while in the background of debates, do lead to significant differences regarding curriculum study, development, implementation, and assessment.


As Herbert Kliebard (2004) makes clear, the historical struggle over the curriculum, including its definition, has involved a variety of competing interest groups. Their debates, sometimes quite heated in the scholarly writings and popular journals of the field, have focused on a number of related, crucial issues, for example the nature and effects of societal forces, the nature of learners and learning, the nature of subject matter, and the purposes of schooling.

One of the fundamental questions of curriculum deliberation, though not always explicitly articulated, involves the extent to which and how the curriculum should respond to social change. For example, throughout the past century various educators and others have expressed concerns about the lessening influence of the family and the church. Should the school be expected to play a more expansive role in children’s lives, such as regarding what is taught in the classroom? Should the curriculum focus on aspects of daily life that other institutions previously could be expected to address, for example involving personal character, physical health, and domestic affairs? Likewise, concerns about economic and technical-scientific changes have convinced some educators that the curriculum should place a high priority on preparing students for work (vocational skills), for utilizing the latest technologies, and for proficiency in mathematics and science. Others have been more concerned with what they perceive to be a significant decrease in civic participation and have advocated a curriculum that places more emphasis on the active and critical engagement of social issues and problems.

Another broad issue involves the extent to which children should be involved in their own learning experiences, i.e., in directly helping to plan them or at least to have their personal interests and concerns providing guidance when planned by others. Some educators have argued that active engagement with activities and materials, so that children initiate and direct (construct) their own understandings, is in fact necessary for authentic learning to take place. If so, then the curriculum would need to be organized with the learners’ experiences uppermost in mind. Standardized content and a top-down design approach would be downplayed and flexibility, creativity and direct student (and teacher) involvement would be expected.
Issues involving the nature of subject matter for schools have also been intensely addressed during the last century. For example, some have argued strongly for curriculum to follow the structures of the recognized and longstanding academic disciplines, building on the work done by recognized scholars over several centuries. Thus, children might study the discipline of history and in essence become “junior historians” at a young age. Others have argued that subject matter needs to be more broadly conceived and made more “relevant” to learners; thus, it is argued, a field like social studies that focuses on citizenship rather than the discipline of history would be more appropriate for a population of students who are not necessarily college-bound. And others have suggested a more integrated approach to curriculum, with multidisciplinary content and skills being brought to bear on projects or social problems that children are studying and that may be of more personal interest to them.

Finally, the larger social purpose of schooling has always been a matter of considerable dispute. What is the primary goal that schools should aspire to? Is it extensive knowledge of the academic disciplines; critical understanding of the social and natural worlds; advanced literacy skills; development of the full range of intelligences; preparation for adult careers; advanced reasoning and problem-solving abilities; active involvement in democratic citizenship; a passion for learning and self-understanding; or some other overall aim? Indeed, to what extent should there even be predetermined goals toward which curriculum decisions should be directed? Such fundamental questions can lead to very different perspectives on what the curriculum is, what it is for, and how it should be determined.

There were at least four interest groups during the first half of the 20th century that competed for supremacy in the determination of the curriculum (Kliebard, 2004). The first group, which held sway on curriculum matters during the late 19th-early 20th centuries, was the humanists, who could be considered “the guardians of an ancient tradition tied to the power of reason and the finest elements of Western cultural heritage” (p. 23). They sought to provide all children with a common curriculum that stressed mental discipline and the powers of reasoning on the one hand and the classical traditions (including religious) and academic (university-based) disciplines on the other. Reacting to this approach were at least three groups of reformers who sought to change what schools taught and how the curriculum was organized. The developmentalists or child-centered progressives were educators who sought curriculum that was more “in harmony with the child’s real interests, needs, and learning patterns” (p. 24). Some adherents, such as William Heard Kilpatrick in 1918, believed that children should not be taught directly but instead should engage in projects that essentially linked their immediate experiences and interests with worthy living. Academic content would be brought to bear when necessary for the fulfillment of “purposeful acts.”

Another group consisted of social efficiency educators or scientific curriculum makers who were particularly concerned with “creating a coolly efficient, smoothly running society,” which included “applying standardized techniques of industry to the business of schooling” (p. 24). Like Franklin Bobbitt, they sought to ascertain, with expanded testing and counseling, the expected futures of children and then differentiate the curriculum so that children would receive the kind of education that would best prepare (fit) them for their predicted life after school. In such a way, schools would be less wasteful and the curriculum could more directly link to the presumed adult roles that future citizens would occupy. Identifying precise objectives for each subject area, creating more school subjects to transmit the knowledge that was needed to live and work in an increasingly industrialized and urbanized nation, and providing different kinds of education for the different potential futures of students were important aspects of this group’s agenda.

A fourth group of educators took a social meliorist or social reconstructionist approach to curriculum work, whereby they “saw the schools as a major, perhaps the principal, force for [progressive] social change and social justice” (p. 24). When in 1932 George Counts asked, “Dare the school build a new social order?” it wasn’t a question of whether it could or not, but whether it would strive to do so. Emphasizing the political character of the curriculum choices that are made, the primary question for these educators was not whether to advocate or not, but the nature and extent of one’s advocacy, not whether or not to encourage a particular social vision in the classroom but what kind of social vision it would be. For these educators it was to be one dedicated to the elimination of poverty, inequality and prejudice.

While other important interest groups could be mentioned, especially for the last half-century (for example, see Michael Apple’s book Educating the “right” way: Markets, standards, God, and inequality, for a discussion of the New Right alliance consisting of neoliberals, neoconservatives, authoritarian populists, and elements of the professional managerial class), the point being made here is that groups of educators and others have long advocated for different approaches to certain fundamental issues of schooling. Their different ideas, policies and practices relating to curriculum have in fact helped to shape the battleground that is our schools today.


Issues of planning and development have been arguably the least contentious aspect of this applied field of study, but in fact here too differing approaches have served as a source of conflict. The dominant approach that was made famous by Ralph Tyler and others has been challenged by those seeking an approach to curriculum design that is based less on linearity, specificity, and observable and measurable performance than on reflective inquiry, flexibility, expressive outcomes, and democratic life.

Probably the most evident source of conflict in schools involves what should be taught to which students at what time, involving issues of scope and sequence. Here, the contentious nature of the debates appears more regularly in the popular media as well as in the work of curriculum theorists and policymakers. This may be of little surprise when one considers the fact that in the state of Florida, for example, 440 viable high school majors were recently approved for school adoption (and, college-style, for students to choose from). Obviously no school can select more than a handful to offer. Who will make these choices, using what criteria? What will make up the actual sequence of courses in each major chosen? What will comprise the content of the courses offered? From animal caretaker, business publishing, cabinetmaking, florist assistant, and eurhythmics, to art and theatre, social studies, English and journalism, mathematics, and biological sciences, the options for what can be studied in school—the universe of knowledge that can be viewed as legitimate for school adoption—is made more explicit here.

The conflicts that rage over the courses to be taught as well as the character of the courses chosen can often be found in bold form in the headlines of local and national newspapers and popular periodicals. What follows are 40 representative examples from the last several years that provide an indication of the wide-ranging debates over content selection that are currently taking place:

• As AP expands, studies disagree on its value
• Panel sounds alarm on science education
• Can less equal more? Proposal to teach math students fewer concepts
in greater depth has divided Maryland educators
• Yoga, hip-hop . . . This is P.E.? Updated programs are more active and varied, but new tests, finances, training, and traditions slow their adoption
• Rethinking recess: As more schools trim breaks, new research points to value of unstructured playtime
• Law tells schools to teach students about online safety
• Computer science fighting for time
• Driver education hits dead end
• High schools teach more kids basics in Finance 101: 14 states require money management to graduate
• Arabic, the new French?: Pressure to compete globally and boost national security is driving interest in less-common languages such as Chinese and Arabic
• U.S. students need more math, not Mandarin
• Traditional social focus yielding to academics: Instead of a year to adjust to puberty, 13-year olds now given algebra and other demanding coursework
• Giving voice to teen’s thoughts: Programs in Miami-Dade and Broward counties give high-schoolers the opportunity to learn through poetry slams and spoken word workshops
• Students set the rules at New York City high school
• High schoolers combine service learning projects with classroom learning
• As the evolution-creationism debate rages, Florida picks a new generation of textbooks
• Intimidation alleged on “intelligent design”: Teacher cites school board pressure
• Ohio board undoes stand on evolution
• Vocational education conflict heating up
• Vocational education: “It’s not your grandfather’s trade school”
• Educators divided over what to learn from 9/11
• Have we forgotten civic education?: Two centuries after Jefferson, social studies are lacking at public schools
• Philly schools to require African history class
• Tennessee creates official curriculum for African-American history: Many schools offer the subject, but now classes will be uniform
• International studies a hard sell in U.S.
• Today’s textbooks labor to be careful with Clinton scandal
• Need to celebrate Constitution Day called into question
• How can we fix the world if we can’t read a map?
• Social Studies losing out to reading, math
• Teaching Thanksgiving from a different perspective
• Clauses and commas make a comeback: SATs helps return grammar to class
• Some teachers say tests stifling creativity: Drilling for exams replaces hands-on learning activities
• Facing obstacles to sex education: Maryland schools reach to parents from different cultures
• Sex-ed class becomes latest school battleground: Some parents and states object to restrictions linked to federal abstinence funds
• Education Commission of the States wants to put arts back on states’ high-priority list
• Democrats in 2 southern states push bills on Bible study
• For teachers, much gray if curriculum adds gays
• Lawsuit in Massachusetts challenges use of gay-themed storybook
• School must teach back-to-basics “phonics”
• Teach the simple joys of reading

One can see from these and other examples that many of the issues discussed earlier in this chapter are embedded in our more public debates, with reference to the purpose of schooling, the nature of learners and learning, the nature of subject matter, and the relationship of schools to social change, as well as issues more directly involving, e.g., standardized testing, legislative and school board activities, the decision making process, scholarly research, and cultural differences. Indeed, the curriculum represents the essence of what schools do—that is, what they teach, or, put more broadly, what experiences they provide for students—and so it is hardly a surprise that this is where the conflicts sometimes rage most intense. Headlines trumpet; politicians legislate; advocacy groups lobby; parents inquire or insist; and teachers and their students are left to work out the possibilities for teaching and learning in their classrooms.

As Fred Inglis (1985) suggests, at a basic level the curriculum is “another name for the officially sanctioned and world-political picture which we produce, circulate and reproduce in our society” (p. 63). It does not merely imply but actually teaches particular versions of not only what is “good” in life and what is not but also who is good and who is not. The curriculum represents a kind of battleground in which contrasting messages of who we are and what we should become, both individually and as a society, are played out. In effect, the curriculum comprises “stories we tell ourselves about ourselves” (p. 31). What “stories,” then, are we telling by the arguments that we have and the choices that we make? Are they ones that emphasize democratic social relations, an expansive view of intelligence, and critical understandings of the social and natural worlds, or do they instead stress competitive individualism, “basic skills,” and preparation for work? It is clear that these are truly stories that matter.


Bobbitt, Franklin. 1918. The curriculum. Boston: Houghton Mifflin.
Eisner, Elliott W. 2002. The education imagination: On the design and evaluation of school
programs. Upper Saddle River, NJ: Prentice Hall.
Inglis, Fred. 1985. The management of ignorance: A political theory of the curriculum. Oxford, Eng.: Basil Blackwell.
Jackson, Philip L., ed. 1992. Handbook of research on curriculum. New York: Macmillan.
Kliebard, Herbert. 2004. The struggle for the American curriculum, 1893-1958. New York: RoutledgeFalmer.
Marshall, J. Dan, James T. Sears, and William H. Schubert. 2000. Turning points in
curriculum: A contemporary American memoir. Upper Saddle River, NJ: Merrill.
Ornstein, Allan C., Edward F. Pajak, and Stacey B. Ornstein, eds. 2007. Contemporary issues in curriculum. Boston: Pearson.
Schubert, William H., Ann L. Lopez Schubert, Thomas P. Thomas, and Wayne M. Carroll. 2002. Curriculum books: The first hundred years. New York: Peter Lang

case examples ~ curriculum evaluation

Be A Creator
[politics of curriculum development and evaluation; context of evaluand]

Sex Ed Curriculum Eval Tool
[components of curriculum evaluation]

Transition Mathematics Evaluation
[planning/designing curriculum evaluation]

What Works Clearinghouse
[evaluation clearinghouses]

Mexican American Studies
[bias in curriculum]




Analyzing Pedagogical Principles in Curriculum
[pedagogy and curriculum]








What learning outcomes really matter? The Case of DARE
[learning outcomes]