This paper examines the properties of two nonexperimental study designs that can be used in educational evaluation: the comparative interrupted time series (CITS) design and the difference-in-difference (DD) design. The paper looks at the internal validity and precision of these two designs, using the example of the federal Reading First program as implemented in a midwestern state.
Using Volunteers to Improve the Academic Outcomes of Underserved Students
School-based mentoring programs have been shown to improve students’ academic performance and self-confidence. This study examines what makes the Big Brothers Big Sisters of America school-based mentoring program effective, offering key insights for practitioners. It also contributes a theoretical structure with which to assess other randomized evaluations of such programs.
This paper presents a conceptual framework for designing and interpreting research on variation in program effects. The framework categorizes the sources of program effect variation and helps researchers integrate the study of variation in program effectiveness and program implementation.
This paper explores the use of instrumental variables analysis with a multisite randomized trial to estimate the effect of a mediating variable on an outcome.
Despite the growing popularity of the use of regression discontinuity analysis, there is only a limited amount of accessible information to guide researchers in the implementation of this research design. This paper provides an overview of the approach and, in easy-to-understand language, offers best practices and general guidance for practitioners.
Using an alternative to classical statistics, this paper reanalyzes results from three published studies of interventions to increase employment and reduce welfare dependency. The analysis formally incorporates prior beliefs about the interventions, characterizing the results in terms of the distribution of possible effects, and generally confirms the earlier published findings.
An Analysis of the Interaction among Quality-of-Life Indicators from the New Communities Program Evaluation
This paper explores analytic methods that assess the rate at which changes in neighborhood quality of life occur. It looks at correlations among quality indicators over time and the effect of both neighborhood context and conditions beyond the neighborhood, like the Great Recession, identifying which indicators are predictors of others.
Evidence from Ten Experimental Welfare-to-Work Programs