With the press to improve the academic achievement of children, educators are increasingly turning to after-school programs as a venue to provide supplemental academic support. The U.S. Department of Education’s 21st Century Community Learning Centers (21st CCLC) program, established in 1999, provides approximately $1 billion to states each year, with the primary purpose of providing opportunities for academic enrichment to help students meet state and local standards in core content areas. Many other after-school programs also provide various forms of academic support.

Findings from a previous national evaluation of the 21st CCLC program indicate that, on average, program grants awarded between 1999 and 2002 had a limited impact on elementary school students’ academic achievement. A possible factor is the finding that most academic activities at the evaluation sites consisted of only homework sessions in which students received limited additional academic assistance. Therefore, the low levels of formal academic assistance offered in these programs and the programs’ limited academic effects highlighted the need for improved academic instruction in after-school settings.

In response, the Institute of Education Sciences funded the development and evaluation of instructional resources for reading and math that could be used in after-school programs for elementary school students. This study tests whether, after one and two years, such instructional approaches produce better academic outcomes than regular after-school services that consist primarily of help with homework or of locally assembled materials that do not follow a structured curriculum. The specific instructional approaches (developed by Success for All for reading and Harcourt School Publishers for math) were adapted from existing curricula used during the regular school day to reflect the particular needs of after-school programs: engaging materials that could be delivered in 45-minute modules that did not require students to attend the program every day.

This project tests one of many possible approaches for supplemental instruction (others might be more experiential or focused on enrichment). Furthermore, it is not a test of the effects of after-school program services compared to no after-school program. As part of the intervention, these instructional models were supported by implementation strategies related to staffing, training and technical assistance, and student attendance. This second and final report presents findings, after two years of program implementation, from a two-year intervention and random assignment evaluation in 27 after-school centers (12 for reading and 15 for math) for students in grades 2 through 5. (A report after the first implementation year examines effects in 50 after-school centers.) The report addresses questions that are relevant to both years of implementation, such as whether one-year impacts are different in the second year of program operations and whether students benefit from being offered two years of enhanced after-school academic instruction.

Key Findings for Math

After two years of implementation, the enhanced math model, Mathletics, had the following findings:

• The enhanced math program was generally implemented as intended (in terms of staff characteristics, training, and usage of instructional materials). As planned, the enhanced math instruction was offered an average of four times a week in 45-minute sessions. In over 95 percent of observed classes, instructors used the Mathletics materials and organized the transitions between the parts of the daily lesson as intended.



• Students in the enhanced program received math instruction that was more structured and intensive than regular after-school program students. The enhanced program produced a 26 to 30 percent increase in hours of academic instruction for math over the course of one school year, and a 22 percent increase over two years.



• One year of enhanced instruction produced positive and, for the first year, statistically significant impacts on student achievement, representing 10 percent more growth over the school year for students in the enhanced program group, as measured by SAT 10 total math scores.



• Two years of the enhanced program produced no additional achievement benefit beyond the one-year impact. Both experimental and nonexperimental analyses support this conclusion.

Key Findings for Reading

Adventure Island, the reading model, had the following findings:

• The enhanced reading program was generally provided by the intended staff who received training and used the instructional materials. As planned, the enhanced reading instruction was offered an average of four times a week for 45 minutes a day.



• Some reading instructors reported in the first year that it was difficult to include all aspects of the reading program and maintain the intended pace of the daily lesson plan. In the second year, interviews confirm that this continued to be a challenge for staff in about half of the after-school centers.



• Students in the enhanced program received reading instruction that was more structured and intensive than regular after-school program students. The enhanced program produced a 22 to 23 percent increase in hours of academic instruction for reading over the course of one school year, and a 20 percent increase over two years.



• After one year, the students in the enhanced reading program did not experience a statistically significant impact on their performance on the SAT 10 total reading test or the DIBELS fluency test measures.



• Two years of participation produced significantly fewer gains in reading achievement for students in the enhanced program group. Both experimental and nonexperimental analyses support this conclusion.