American Educational Research Association

Apr 14, 12:25-13:55h

Katherine Drake and Cheryl Carlstrom, Saint Paul Public Schools

“Evaluation of Supplemental Educational Services in a Midwestern Urban School District”

Supplemental Educational Services (SES) is defined as tutoring and other supplemental academic enrichment provided outside of the school day that is specifically designed to help students achieve proficiency on state academic standards as measured by the state’s assessment system. In compliance with the No Child Left Behind Act of 2001, any Title I school or district identified as not making Adequate Yearly Progress (AYP) for three or more consecutive years must offer SES to all low-income students attending the school. While the U.S. Department of Education requires that SES providers demonstrate effectiveness in improving student achievement, states and districts have limited capacity to monitor providers and to evaluate their performance.

To date, research on SES has not consistently shown a demonstrable impact on student achievement (Authors, 2007a; Authors, 2007b; Authors, 2010). The research, evaluation, and assessment office of a Midwestern urban school district conducted an evaluation in order to measure the effectiveness of SES providers at improving the academic achievement of students who received service in that district during the 2009-10 school year. The study was designed to answer three questions: 1. Who participated in SES? 2. What was the impact of SES on student achievement? 3. How did SES providers compare in terms of student achievement outcomes?

Data from a supplemental service database that included provider, session type, and attendance information for 1,692 registered students were linked to student demographic data as well as to fall 2009 and spring 2010 Measures of Academic Progress (MAP) reading and math data and to spring 2009 and spring 2010 data from the statewide tests in reading and math data. After reviewing participant demographics overall, by school, and by provider, we compared the achievement test scores of SES students to SES-eligible students (i.e., all students who received free or reduced price lunch and enrolled in an SES school) who did not register for SES.

The participant file was split by subject area in which each student received service (math or reading), with some students receiving assistance in both subject areas. Propensity Score Matching was used to identify comparison groups (by subject area) from the pool of all 2009-2010 non-participating SES-eligible students. Analysis of MAP and statewide test data supports existing research findings of little to no difference in test performance of students who receive SES compared to those who do not. Where statistically significant differences were found, matched students who did not receive SES outperformed those who did. When hours of service were considered, students who received 20 or more hours of SES met achievement test targets at the same rate as those who participated in fewer than 20 hours. While the percent of students achieving proficiency on the statewide assessment, in both math and reading, did vary by provider, no single provider showed success across all measures and subject areas.

In conclusion, this evaluation confirms the lack of support for SES found by researches in other urban school districts.