Assessing an intervention’s effects on multiple outcomes increases the risk of false positives. Procedures that make adjustments to address this risk can reduce power, or the probability of detecting effects that do exist. MDRC’s Reflections on Methodology discusses how to estimate power when making adjustments as well as alternative definitions of power.
To improve outcomes among high-interest borrowers, policymakers need to understand what is driving usage. This second post in MDRC’s Reflections on Methodology series discusses how a data discovery process revealed clusters of borrowers who differed greatly in the kinds of loans and lenders they used and in their loan outcomes.
Machine learning algorithms, when combined with the contextual knowledge of researchers and practitioners, offer service providers nuanced estimates of risk and opportunities to refine their efforts. The first post of a new series, Reflections on Methodology, discusses how MDRC helps organizations make the most of predictive modeling tools.
A Primer for Researchers Working with Education Data
Predictive modeling estimates individuals’ probabilities of future outcomes by building and testing a model using data on similar individuals whose outcomes are already known. The method offers benefits for continuous improvement efforts and efficient allocation of resources. This paper explains MDRC’s framework for using predictive modeling in education.
A Guide for Researchers
Conducting multiple statistical hypothesis tests can lead to spurious findings of effects. Multiple testing procedures (MTPs) counteract this problem but can substantially change statistical power. This paper presents methods for estimating multiple definitions of power and presents empirical findings on how power is affected by the use of MTPs.
This paper provides a detailed discussion of the theory and practice of modern regression discontinuity. It describes how regression discontinuity analysis can provide valid and reliable estimates of general causal effects and of the specific effects of a particular treatment on outcomes for particular persons or groups.
This paper provides practical guidance for researchers who are designing studies that randomize groups to measure the impacts of educational interventions.
In a speech before the Association for Public Policy Analysis and Management Conference on November 7, 2008, Judith M. Gueron, President Emerita and Scholar in Residence at MDRC, accepted the Peter H. Rossi Award for Contributions to the Theory or Practice of Program Evaluation.
This MDRC working paper on research methodology explores two complementary approaches to developing empirical benchmarks for achievement effect sizes in educational interventions.
This MDRC working paper on research methodology provides practical guidance for researchers who are designing studies that randomize groups to measure the impacts of interventions on children.