Conducting strong research studies is a prerequisite for evidence-based programs and policy, but interpreting study findings can be challenging. For example, views differ in how to interpret a result that is not “statistically significant.” Does that mean an intervention had no effect at all? Likewise, in a study where only some findings are statistically significant, often only the ones that are statistically significant are highlighted while others are ignored or downplayed. Is that the right approach?

Despite these types of ambiguities, policymakers, funders, and practitioners must make decisions on expanding an intervention with promising effects, on abandoning or strengthening an intervention with weak effects, or on doing more research to arrive at a clearer set of findings. This document compares two statistical approaches to interpreting impact findings that can help in making decisions like these: a conventional approach that uses statistical significance and that has been the standard approach in social program evaluation for decades, and a Bayesian approach that offers a more intuitive way of interpreting impact estimates.

Both approaches require expertise in producing and interpreting findings. However, the Bayesian approach can provide direct answers to the questions that programs, funders, and policymakers often have: Does this program work? How confident can one be that effects exceed meaningful thresholds? The questions that the conventional approach can answer are generally less useful for making policy decisions, which suggests that evaluators can help decision makers extract more value from rigorous research by providing Bayesian interpretations of impact estimates.

Although there are advantages to Bayesian methods, they have not been used often in social science research, and it may not be clear to program leaders, funders, and policymakers how to interpret results from a Bayesian analysis. This document is intended to fill that gap and to provide information on how the conventional and Bayesian approaches can answer a seemingly simple question: Is an intervention effective?