AERA 2012: Evaluation of Supplemental Educational Services in a Midwestern Urban School District

American Educational Research Association

Apr 14, 12:25-13:55h

Katherine Drake and Cheryl Carlstrom, Saint Paul Public Schools

“Evaluation of Supplemental Educational Services in a Midwestern Urban School District”

Supplemental Educational Services (SES) is defined as tutoring and other supplemental academic enrichment provided outside of the school day that is specifically designed to help students achieve proficiency on state academic standards as measured by the state’s assessment system. In compliance with the No Child Left Behind Act of 2001, any Title I school or district identified as not making Adequate Yearly Progress (AYP) for three or more consecutive years must offer SES to all low-income students attending the school. While the U.S. Department of Education requires that SES providers demonstrate effectiveness in improving student achievement, states and districts have limited capacity to monitor providers and to evaluate their performance.

To date, research on SES has not consistently shown a demonstrable impact on student achievement (Authors, 2007a; Authors, 2007b; Authors, 2010). The research, evaluation, and assessment office of a Midwestern urban school district conducted an evaluation in order to measure the effectiveness of SES providers at improving the academic achievement of students who received service in that district during the 2009-10 school year. The study was designed to answer three questions: 1. Who participated in SES? 2. What was the impact of SES on student achievement? 3. How did SES providers compare in terms of student achievement outcomes?

Data from a supplemental service database that included provider, session type, and attendance information for 1,692 registered students were linked to student demographic data as well as to fall 2009 and spring 2010 Measures of Academic Progress (MAP) reading and math data and to spring 2009 and spring 2010 data from the statewide tests in reading and math data. After reviewing participant demographics overall, by school, and by provider, we compared the achievement test scores of SES students to SES-eligible students (i.e., all students who received free or reduced price lunch and enrolled in an SES school) who did not register for SES.

The participant file was split by subject area in which each student received service (math or reading), with some students receiving assistance in both subject areas. Propensity Score Matching was used to identify comparison groups (by subject area) from the pool of all 2009-2010 non-participating SES-eligible students. Analysis of MAP and statewide test data supports existing research findings of little to no difference in test performance of students who receive SES compared to those who do not. Where statistically significant differences were found, matched students who did not receive SES outperformed those who did. When hours of service were considered, students who received 20 or more hours of SES met achievement test targets at the same rate as those who participated in fewer than 20 hours. While the percent of students achieving proficiency on the statewide assessment, in both math and reading, did vary by provider, no single provider showed success across all measures and subject areas.

In conclusion, this evaluation confirms the lack of support for SES found by researches in other urban school districts.

AERA 2012: Parents’ Perspectives on Privatizing Trends in Education

American Educational Research Association

Apr 15, 14:15-15:45h

Patricia Burch and Rudolfo Acosta (Univ of Southern California)

“Where Do I Go? Parents’ Perspectives on Privatizing Trends in Education”

Introduction and Rationale: There are dramatic changes underway in the Federal role in increasing access and opportunity for students living in poverty. On the one hand, the federal government has become increasingly proactive in directing instruction at the Federal, state and local levels. For example, the No Child Left Behind Act (the 2001 reauthorization of the Elementary and Secondary Education Act) tied federal funding for economically disadvantaged communities to test score performance and introduced progressive sanctions for schools. Standing partially in tension with this centralization, private engagement in the governance and administration of public education is expanding and evolving.

Central Arguments: In this paper, the authors make six central arguments about the ways in which the role and influence of for profit firms in k-12 public education is changing.

  1. Large corporations—and the laws that protect them—increasingly drive how and what the public learns about education.
  2. With government programs being slashed, companies are turning to new money sources in order to expand.
  3. Education companies are using relationships with school districts to get teachers, students and parents hooked on their products.
  4. For profits are pushing legal boundaries by using not for profits as marketing arms.
  5. When it comes to private sector involvement in public education, there is no accountability
  6. The changes described follow a general pattern. However, whether and how government agencies and private industry trade places varies depending on the setting and what is being sold.

Methods: This research is part of an ongoing multisite mixed methods study on the implementation and impact of supplementary education services. Data were collected from five urban school districts representing a variety of student demographics in: Milwaukee, Wisconsin; Minneapolis, Minnesota; Chicago, Illinois; and Austin and Dallas, Texas. This paper draws on findings from the qualitative portion of the study specifically aimed at analyzing which factors influence parent or student choices in selecting supplemental education providers. Data collected consists of focus groups with parents (n=174) of students eligible to receive and/or currently receiving supplementary services. Two focus groups, approximately 1.5 hours each took place at each of the sites with translation offered in Spanish, Hmong, and Somali. The racial and ethnic demographics of parents consisted of the following: White, Latino, African American, African, Asian, Biracial, Native American, Multiracial, or other. Parents in the sample had children eligible for services in elementary, middle, or high school. To delineate the social construction of parents as the targets of policy, a textual analysis of the federal NCLB policy concerning the implementation of supplementary services was used.

Contributions: The paper provides voice for perspective of those at the receiving line of privatization, in particular students with disabilities and English language learners. Based on these voices, we identify the core issues that policymakers must wrestle with if current forms of privatization are to strengthen ties between parents and schools. We also show how Federal education policy has helped to legitimize these changes.

AERA 2012: Preliminary Findings of a Multisite Study of the Implementation and Effects of Supplemental Educational Services

American Educational Research Association

Apr 16, 14:15-15:45h

Patricia Burch (Univ of Southern California) and Carolyn Heinrich (Univ of Texas – Austin)

“Preliminary Findings of a Multisite Study of the Implementation and Effects of Supplemental Educational Services”

Under No Child Left Behind (NCLB), schools that have not made adequate yearly progress in increasing student academic achievement for two years or more are required to offer parents of children in low-income families the opportunity to receive extra academic assistance, or supplemental educational services (SES). This paper works with ideas from the field of policy sociology to develop an understanding of SES as shaped by a complex interplay of forces; this field is particularly useful in analyzing a complex policy such as SES because it seeks to understand the intersection of macro and micro systems. This multilayered policy requires these various systems and actors to interact in new ways and in relatively uncharted policy territory. Research Setting and Design Findings are based on data from an ongoing multi-site study of the implementation and impact of supplemental educational services. The central purpose of this study is to understand whether and how providing students with academically focused out-of-school tutoring in reading and mathematics contributes to improvements in their academic performance, specifically in reading and mathematics. It involves three linked phases of research. Phase 1 is an in-depth qualitative study designed to define key elements of SES program models and the policy and practice variables that mediate implementation of these models and to also inform the construction of the measures of SES treatment for quantitative analysis. Phase 2 is a quantitative study investigating selection into SES (i.e., who registers and participates) and SES program impacts, using propensity score matching and fixed-effect methods with nonequivalent (internal) comparison groups. Phase 3 is a follow-up qualitative study to examine whether program features identified in Phase 1 continue over time and to further inform our interpretation of the quantitative findings of program impact from Phase 2. We are conducting this research in five urban school districts located in four states and representing different student demographics. Findings Our analysis reveals three primary themes. First, in order to understand and address the policy problems in SES, we need to see the policy as driven by a particular theory of action – the idea that public policy needs to be modeled around and accommodate the market place. Second, while NCLB employs the common rhetoric to equalize educational opportunities for the poor, the design of SES contributes to power asymmetries that sometimes privilege the financial interests of individual private firms and frustrate collective access to information. Thus, expanding the ability of SES to serve the goal of equal opportunity requires addressing the ways in which design and discourse of the policy denies equal access to knowledge for eligible participants and stakeholders. The other key to improving SES lies in the quality of instruction inside the SES classroom and the relationships between teachers, parents, students and tutors. In short, any effort to improve SES must address the perverse or inadequate incentives written into the design of current policy while solutions for improving policy should be anchored in deep understanding of actual classroom/tutoring conditions and the quality of relationships built there.

AERA 2012: Black Women’s Community Othermothering and Supplementary Education

American Educational Research Association

April 15, 8:15-9:45h

Amira Millicent Davis

“‘Educate a Woman and You Educate a Nation’: Black Women’s Community Othermothering and Supplementary Education”

This paper is of a yearlong qualitative ethnographic study of women of African descent in the U.S. and UK who provide supplementary education programs in their communities. Data was collected through interviews, primary and secondary sources and social media. The cultural work of these women is interpreted through the lens of maternal activism articulated in Hill-Collins community and othermothering in which Black women’s epistemologies are privileged. The women presented have created sustainable community education spaces that simultaneously enhance academic performance, wage resistance, perpetuate traditions and rituals and preserve cultural knowledge. The goal of this work is to share their experiences as models for community-based literacy programs, educational advocacy and individual and community empowerment with activists, practitioners, scholars, funders and policymakers.

UBC Workshop: Reconstruction after Japan’s Triple Disaster: Lessons for British Columbia

My colleague from UBC’s Geography Department, David Edgington, has put together a terrific workshop with funding from the Japan Foundation to discuss lessons from the aftermath of the triple disaster of the Great East Japan Earthquake, tsunami, and nuclear crisis.

Presentations on March 15


Hirokazu Tatano, Disaster Prevention Research Institute, Kyoto University: “The Great Eastern Japan Earthquake and Tsunami: Implications for Infrastructure Planning and Management”

Panel 2

Theme: Learning from Earthquakes and Tsunamis in Japan and North America

Hirokazu Tatano, Disaster Prevention Research Institute, Kyoto University “The Japanese Emergency Management System”

John Oakley, Snr. Regional Manager, Emergency Management, BC “The British Columbia Emergency Management System”

Moderator: Ilan Vertinsky, IAR and Sauder Business School, UBC.

Panel 3

Theme: Community Reconstruction after Disasters

Masahige Motoe, Architecture and Urban Design Course, Tohoku University “Housing Transitions after the Japan Quake and the Activities of Architects – ArchiAid

Julian Dierkes, Director, Centre for Japanese Research, IAR, UBC “Small and Medium Education Businesses in the Economic Recovery of Tohoku”

Moderator: Chihiro Shimizu, International School of Economics and Business Administration, Reitaku Univeristy; Advisor to the Ministry of Land, Infrastructure, Transport and Tourism, Japan.

Panel 4

Theme: Film: Social Media and Disasters

Jamie Williams, School of Journalism, UBC “The Evolving Role of Social Media in Emergency Management”

Peter Anderson, School of Communications, Simon Fraser University “The Evolving Role of Social Media in Emergency Management”

Moderator: David Rummel, Visiting Professor at the School of Journalism, UBC; Senior Producer for News and Documentary at the New York Times.

Presentations on March 16

Panel 5

Theme: Economic Reconstruction after Disasters

Stephanie Chang, School of Community and Regional Planning, UBC “Economic Impacts and Recovery in the Great East Japan Earthquake Disaster

Glen Magel, Director, Safety and Security, BCIT; Board Chair, Emergency Preparedness for Industry and Commerce Council “EPIC: The Emergency Preparedness for Industry and Commerce Council

Moderator: David W. Edgington, Geography, UBC

Panel 6

Theme: Universities and Cities in Disasters

Masahige Motoe, Architecture and Urban Design Course, Tohoku University “The Reconstruction Process at the Tohoku University Campus

Ron Holton, Chief Risk Officer, Risk Management Services, UBC “Enhancing Disaster Response Preparedness at UBC

Moderator: Kevin Wallinger, Director, Office of Emergency Management, City of Vancouver

Another Academic Job Focused on Supplementary Education?

Perhaps I’ve found a second job that is advertised with a focus on supplementary education, though I have the strong suspicion that Marburg University in Germany has something else in mind when they are looking for a professor who works on “extramural education of youth” (außerschulische Jugendbildung). I suspect that this has more to do with community centres that offer specific programs for youth or perhaps programs that focus on youths who are not attending school…

First Academic Job Ad to Specify Supplementary Education

As a sign of the growing institutionalization of research on supplementary education and the leadership of Prof. Mark Bray (Hong Kong Univ) in this field, the Comparative Education Research Centre at the University of Hong Kong is advertising a position as Full-time Research Assistant Professor in Comparative Education with a particular focus on Shadow Education at the University of Hong Kong:

Applicants are invited for appointment as Research Assistant Professor in Comparative Education in the Faculty of Education, as soon as possible for a period of three years.

Applicants should possess a Ph.D. degree with relevant research background and demonstrate ability to produce published work. The appointee will work under the supervision of Professor Mark Bray within the framework of the UNESCO Chair in Comparative Education which has been established at the Faculty level. This Chair focuses on social inequalities and access to educational opportunities, especially the nature and implications of Shadow Education (private supplementary tutoring). The appointee will join a team and take responsibility for either a global emphasis or a focus on a particular region or country in comparative perspective. He/She will work with colleagues in the Faculty including the Comparative Education Research Centre (CERC) on other dimensions in the field of Comparative Education, and will develop research proposals for future projects. Teaching responsibilities will be within the broad domain of education and international development at undergraduate and/or postgraduate levels.

Information about the Faculty and CERC can be obtained at and Enquiries about the post should be sent to Professor Mark Bray, Faculty of Education (e-mail:

A highly competitive salary commensurate with qualifications and experience will be offered, in addition to annual leave and medical benefits. The appointment will attract a contract-end gratuity and University contribution to a retirement benefits scheme, totalling up to 15% of basic salary.

Applicants should send a completed application form and an up-to-date C.V. to Application forms (341/1111) can be obtained at Further particulars can be obtained at Closes May 10, 2012.

The University thanks applicants for their interest, but advises that only shortlisted applicants will be notified of the application result.

This is a fantastic opportunity for a younger scholar, but also for our emerging field!

Book Review: Mary Brinton, “Lost in Transition. Youth, Work, and Instability in Postindustrial Japan”

I recently published a book review in Economic Sociology – The European Electronic Newsletter 13 (2), March 2012: 50-51. While the book doesn’t directly discuss juku, it is highly relevant to an understanding of the educational and career trajectories that contemporary Japanese youth pursue.

Book: Mary C. Brinton, 2011: Lost In Transition. Youth, Work, and Instability in Postindustrial Japan. Cambridge: Cambridge University Press.

In the comparative welfare state and varieties of capitalism literatures, Japan has played a curious role. Its rapid post-war growth entitled it to membership in the OECD and inclusion in purportedly widely-applicable theories about labor, industry, the (welfare) state and interlinkages between these elements that were assumed to constitute a “normal” developed market economy. Some elements of Japanese capitalism endured as distinctive features in many different middle-range theories and their application. The transition from school to work that Mary Brinton writes about with such depth of knowledge is one of these distinctive features.

Brinton focuses on the cultural, social, and human capital carried by organizations rather than individuals. The transition to work is highly structured and involves taken-for-granted understandings of the role of the student, school, and prospective employer. These understandings specifically emphasize the role of the school as a broker in placing students. The central question of the book becomes whether this brokering role has been made obsolete by the end of the labor shortages of the high-growth era and what the school-to-work transition looks like in post-industrial Japan. The surprising answer that Brinton provides is that the institutionalized roles of schools in brokering employment offers continues to serve students in vocational secondary schools well, but it is students at the middling to lower-ranked academic high schools that are turning into the “lost generation” that academics, commentators and policy-makers are increasingly concerned about in Japan.

The book makes a great virtue out of the fact that it resulted from a multi-year process of different research projects that were somewhat interwoven around the central theme of the school-to-work transition from the mid-1990s until the late 2000s. The evidence presented is based on a multi-method approach that is not only convincing in providing readers a glimpse at similar empirical questions from different perspectives, but also in offering a portrayal of the contemporary situation that seems as complete as it could be in just under 200 pages.

The opening chapter sets the stage by discussing the Japanese discourse on the “lost generations” that resulted from several years of a very low intake of new employees into the most desirable and stable jobs in the Japanese economy. Because several cohorts of the mid-1990s faced general hiring freezes at their single point of entry to stable employment, these cohorts are moving through the lifecourse with a significant bulge of unemployment or underemployment, lower job security, fewer benefits and all the social, psychological, and economic challenges that attend the status of being a “lost generation”.

The second chapter discusses the historical roots and institutionalization of the school-to-work transition as it emerged to address severe labor shortages during Japan’s high-growth period. Chapter 3 focuses on the extent to which not just the transition to work, but the entire employment trajectory as it is experienced by men in Japan revolves around attachment to a specific context, or ba. The following chapters continue this focus on the institutional context of the transition to work and present data from a variety of angles including an extended argument for why participants place such great trust in the institutionalized employment system. Chapter 6 as the final empirical chapter presents the life histories of three young men as they have experienced their membership in the lost generation. The conclusion then refocuses insights about the school-to-work transition on the growing awareness of socio-economic inequality in Japan.

The great merit of Brinton’s model is her ability to adapt prominent, predominantly North American theoretical concepts from the sociology of work and education to the particular context of Japanese employment relations. For example, she repeatedly returns to questions raised by Mark Granovetter’s strength of weak ties argument and examines it in the Japanese context.

As I progressed (easily, for it is well-written) through the book, my anticipation continued to build as to what other interesting data Brinton would be able to analyze. Data sources stretch from the census level to illustrate the portrayal of the “lost generation”, to smaller scale surveys that Brinton conducted jointly with some of the most prominent contemporary Japanese sociologists. Because her data collection and conceptualization of her analyses were interwoven with the social scientific discourse in Japan, and perhaps also because this book was originally published in Japanese and thus aimed to connect with this discourse more explicitly than many works, Brinton does an exceptional job at bridging scientific debates between the North American and Japanese contexts.

Brinton is not shy about “revealing” the sometimes haphazard routes by which data presented themselves to her. The story she recounts on pp. 55-56 of how she happened to come into possession of the entire trove of job offers in a local employment office was not only a light-hearted but telling insight into the difficulties of obtaining data. This will be a welcome pointer to some of the graduate students who will undoubtedly read this book that good things will come to researchers who engage a topic with in-depth fieldwork in the actual context of their chosen topic.

I found some aspects of Brinton’s argument less convincing than the overall thrust and structure of the presentation. For example, I am not sure that we need yet another version of what seems like a definition of “institution” in another context, namely Brinton’s use of the term “ba”. While this is a term with many complex connotations that I also encounter in my research on supplementary education in Japan, something as simple as “institutional context” would have served Brinton well. The life histories presented in Chapter 6 do round out the mix of methods employed by including in-depth interviews, but they seem to add very little to the overall argument.

I will be relying on the central empirical chapters of this book in an upcoming seminar on economic and social change to examine education(al policy) as a crucible of the organization of work and society in the Asia Pacific myself and recommend this book not only to readers interested in the specifics of the Japanese case, but to the broader audience of scholars working on employment systems and the welfare state. Brinton will provide you with an engaging overview of the Japanese employment system, but also many insights into the operation of social institutions and individuals’ choices in the context of this system.

Obsessing about Finland

Does the (success of) the Finnish education system have any relevance for Japan?

The degree to which some of my interlocutors in Japan obsess about Finland and see Finnish education as a panacea for all that is wrong (presumably, but see my argument for the central importance of perceptions) with Japanese education has puzzled me for some time. The somewhat less urgent interest in Singapore falls in the same category, I think.

Now, Keita Takayama who has been doing some terrific work on the place of Japanese education in a comparative and global context, makes the case that the reference nation status of Finland (and the lack of such status for E Asian education systems that perform equally well in PISA and similar comparisons) is due a) to the proximity of Finnish policies to OECD precepts, and b) to Finnish pre-PISA status as a “tabula rasa” among education systems that is not burdened by any of the preconceptions about E Asian education systems. Takayama makes this case in an Asia Pacific Memo on February 16, 2012.

Takayama makes his case very succinctly, but Finland’s status as a reference nation has come up in N American debates as well.

See for example:

If I were to start from the premise that there are things about Japanese education that could use reform, what countries would I look to? Or, what countries might I not look to?

I do think that education systems vary significantly by scale. According to Eurostat, the total number of primary and secondary students in Finland is about 1.2 mio. Japan’s comparable figure is 18 mio. This reflects the ratio of the two populations with about 5.3 mio Finns in 2010 and 127 mio Japanese. Clearly, an education system that is fifteen times as large in terms of the number of students enrolled involves different complexities than a smaller counterpart. That is not to say that there aren’t aspects of Finnish education that are worth examining in considering reforms for Japan, I’m just not convinced that Finland is the most relevant example in this regard.

Why not pick more proximate comparators? While South Korea is not even half as populous as Japan its education system operates on a scale that is much closer to Japan. Likewise France which would be an example of an education system that has some structural similarities (high degree of centralization, for example) and operates on a similar scale, though a comparison between Japan and France on comparative educational achievement would likely lead to a large-scale visitation movement from France to Japan, not the other way around.

What about Canada? Again, not quite on the population scale of Japan, but similar success to Japan…

I see the interest in Finland as primarily a faddish one will raise some questions that are wonderful raise about education elsewhere (including in all the countries mentioned in the above), but will not provide panacea.


Reflecting on Tohoku Trip

If you follow me on twitter you know that I recently spent some days in Tohoku  with the intent of looking at post-disaster recovery from the perspective of juku as a small service-oriented business. I am now mulling over some of my observations from this trip in preparation for a March 15 presentation in a workshop organized by my colleague David Edgington at UBC.

During my visit, I was able to meet with 1o juku operators in the region, from Iwate and Miyagi Prefectures to Fukushima.

Here is a loose list of some of the things I will talk about:

  • in terms of the situation that juku find themselves in, the clearly most significant determinant of this situation is whether the operator has access to a teaching space, i.e. whether his/her home/teaching space had been destroyed
  • differences in the impact of the triple disaster by geographic zones:
    • coastal regions: tsunami impact severe, but it largely ends at where ever the water rose to
    • inland: some damage to buildings, but only sporadic
    • Fukushima: radiation impact pervasive
  • relatively few juku operators have abandoned the business
  • while economic opportunities were limited in the country-side pre-disaster, even rural Tohoku was not in as dire a situation (in terms of economic decline, out-migration) as I’ve seen in Shimane Prefecture, for example
  • while there were many casual throughout the region, the number of children who perished was limited and there are also many families who were not impacted materially to any great extent. This has meant that for many juku, business is down (in terms of the number of children taught), but not dramatically so
  • juku are eligible for reconstruction funds as small businesses and these public monies are appropriate to the reconstruction needs of a small service business
  • there is clearly a disaster bubble (震災バブル) in progress, evidence of which could be seen in the many (truck) traffic jams, busy eating establishments, and reflections by locals
  • while the physical clean-up continues apace and the situation for small business seems to be normalizing, there is massive human suffering in evidence all around
  • relief and support efforts are fraught with traps. Offer free tutoring for local students? You’re killing local business opportunities. Offer subsidies for local businesses, non-local chains, etc. will also be eligible.
  • some fascinating volunteer projects in place that are trying to make the best of a very difficult situation and are innovative in doing so
  • the operators of small juku overwhelmingly reported that their corporate cousins (大手塾) have abandoned the region

Some of these observations will come as no surprise to colleagues who have conducted research on post-disaster areas before and I am still trying to organize these thoughts in a more coherent fashion.