The Reading First Impact Study
Overview
The Reading First Program, established under the No Child Left Behind Act of 2001, represents one of the most direct and intensive efforts by the federal government to influence instructional practice and student achievement in low-performing schools. Reading First is predicated on research findings that high-quality reading instruction in the primary grades significantly reduces the number of students who experience difficulties in later years.
The Reading First program used a rigorous application and review process to distribute over $900 million during a five-year period to state and local education agencies for use in low-performing schools with well-conceived plans for improving the quality of reading instruction. The federal funding had to be applied to reading curricula and teacher professional development activities that are consistent with scientifically-based reading research — that is, that incorporate the five critical building blocks of effective reading instruction (phonemic awareness, decoding/word attack, reading fluency, vocabulary, and comprehension). Once approved for funding, schools were expected to: (1) ensure that research-based reading programs and materials are used to teach students in K-3, (2) increase access and quality of professional development of all teachers who teach K-3 students, to ensure that they have effective skills for teaching reading, and (3) help prepare classroom teachers to screen, identify, and overcome barriers to students’ ability to read on grade level by the end of third grade.
Additional Project Details
Agenda, Scope, and Goals
The No Child Left Behind Act required that the U.S. Department of Education contract with an independent organization to conduct a rigorous five-year evaluation of Reading First. Through a competitive process, the U.S. Department of Education selected MDRC and Abt Associates to head a team of research organizations to conduct the Reading First Impact Study.
As specified by the Act, the evaluation addressed the following questions:
- What is the impact of the Reading First program on students' reading achievement?
- To what extent does Reading First result in the implementation and use of the five essential components of reading instruction that have been identified in other scientifically-based research?
- How does variation in the implementation and use of scientifically-based reading instruction influence the impact of such instruction on student reading achievement?
The research team conducted the Reading First Impact Study in two phases. Phase I, the site selection phase, was headed by MDRC. Begun in 2003 and completed in July 2004, it resulted in the identification of more than 250 schools to participate in the study. Phase II, the data collection and analysis phase, was headed by Abt Associates in partnership with MDRC and several other independent research organizations. This phase of the study began in the fall of 2004 and continued over three academic years (2004-2005, 2005-2006, 2006-2007).
Design, Sites, and Data Sources
The Reading First Impact Study uses regression discontinuity analysis to assess the program’s impact on student reading achievement and classroom instructional practices. This analytic approach has a long history, and program evaluators have revived its use to measure effects when random assignment is not appropriate or feasible.
To carry out this design, the research team sought states and school districts that used explicit empirical criteria to select schools for Reading First funding. In each of the participating sites, the state or the local school districts rated candidate schools according to quantitative indicators of their need for and/or ability to benefit from the program. The schools were then ranked in order of priority for funding. Based on the resources available, all eligible schools above a pre-specified cut-off point in the ranking were slated for funding, while schools below that cut-off point were excluded from receiving Reading First grants. The impact analysis compares outcomes for schools that did or did not receive funding. Because the analysis incorporates the school selection process (represented by the rating and ranking criteria) directly into the comparison of schools, it can yield an unbiased estimate of program impacts.
The study includes more than 250 elementary schools in more than 30 school districts in 13 states. The sample consists of schools that were rated by their districts or states as eligible and appropriate for Reading First funding, as well as schools that were rated very similarly but were not able to receive a grant because of funding limitations. Within each district, between 6 and 22 schools, divided equally among Reading First and non-Reading First schools, participated in the study. The study team collected information on students and classrooms in grades 1-3 over a period of three school years.
A national reading comprehension assessment was used to measure student reading achievement. Classroom reading instruction was assessed through classroom observations and surveys of principals, reading coaches/specialists and teachers. District-level student records yielded data on attendance, mobility, and prior achievement.
