Using Evidence as the Driver of Policy Change

The Next Steps in Supporting Innovation, Continuous Improvement, and Accountability


By Gordon Berlin

Good afternoon, Chairman Hatch, Ranking Member Wyden, and members of the Committee.

The federal government spends billions of dollars on policies and programs designed to ameliorate poverty; increase employment, earnings, and income; invest in education; and ensure America’s competitive position in a technologically advancing world. But to insure a real return on these investments for both taxpayers and beneficiaries, we have to do things that actually work.

Over the last decade and a half, the legislative and executive branches have quietly forged a bipartisan consensus around the need to build evidence of effectiveness that would ensure high rates of return on investment for the nation’s social programs. The establishment of the new Evidence-Based Policymaking Commission is only the most recent example of this consensus.

This bipartisan commitment to evidence is paying off — we are beginning to learn what works.  The challenge ahead is twofold: Learning what works at scale and insuring that evidence-based programming is used in practice.

In my written testimony, I offer a brief history of the federal government’s fruitful investment in evidence-building, which really began with the states being the engine of innovation under welfare waivers in the 1980s and 1990s. In more recent years, Congress and both the Bush and Obama Administrations have made an unprecedented effort to embed evidence-building into federal legislation and federal agency budgets, establishing set-asides to fund research, demanding the use of reliable research designs, and creating tiered funding that rewards programs with the strongest evidence, that tests promising models, and that sponsors innovation.

Perhaps the largest evidence-based program of the era is the Maternal, Infant, and Early Childhood Home Visiting program, which provided $1.5 billion in funding for home visiting programs over five years. This legislation, largely crafted by this Committee, was a model. There are several elements worth emphasizing:

  • Prior evidence was used to influence how federal funds could be spent, making it more likely that the funds would make a difference for families.

  • The legislation recognized that there were areas where home visiting was not as effective as desired and offered states funds to test innovative approaches.

  • Funds were set aside for research, including a rigorous national evaluation, to make sure that learning continued and could influence future realizations of home visiting.

Congress has continued this focus on evidence in more recent legislation, including the Workforce Innovation and Opportunity Act of 2014 (WIOA) and the Every Student Succeeds Act of 2015 (ESSA). And both laws also contain provisions for the use of pay-for-success or social impact bond (SIB) initiatives, a promising vehicle for evidence-building — as does the pending Social Impact Partnership Act (SIPA), which would authorize $300 million to support states and localities in the use of social impact bonds to tackle social and public health challenges.

In my remaining time, I’d like to offer some observations about the future of evidence-building, broadly and then with regard to the role of pay-for-success initiatives.

To make the most of these next-generation investments in evidence, we will need to address several issues:

  1. Create a culture of continuous improvement: Rather than being focused on up-or-down judgments about programs or policies, government must develop incentives for using research evidence to make programs more effective over time — building on what we know to make programs stronger, avoiding constantly reinventing the wheel, and changing direction only after continuous improvement doesn’t yield results.

  2. Bolster agency-directed research capacity while supporting pay-for-success innovation: Both vehicles will build evidence, but they have different strengths. The agency research model is preferable when the goal is to learn what works for a broad and persistent problem, where one wants to develop a reliable body of evidence. To support that work, Congress could authorize federal agencies to set aside at least 1 percent of existing program funds for evaluation, a solution that is budget-neutral. On the other hand, the SIB model, with its focus on individual deals, is best used when the goal is to encourage local innovation tied to strong evidence-building and as a means of equipping local government with the tools to assess effectiveness.

  3. Improve access to data for more efficient evaluations, while protecting confidentiality. Private industry is far ahead of the public sector — exploiting big data to understand customer desires, track trends, and assess performance, so that low-performing business units can learn from high-performing ones. The public sector has the data to build comparable systems to enhance program performance, yet federal and states agencies (and their contractors) cannot regularly access and share data for evaluation purposes. My written testimony offers some thoughts about how we can do better.

  4. Clarify federal authority to conduct research. Inconsistencies in federal authority to conduct independent research and evaluation as well as procurement and contracting rules pose additional hurdles for efficient evidence-building.

The social impact bond (SIB) vehicle is a valuable tool for stimulating experimentation, innovation, and the scaling of promising approaches at the state and local level — and the SIPA bill is an important step forward in the effort to support new deals. But SIBs are not a substitute for a comprehensive agency-directed research agenda, which is essential if evidence is to be harnessed effectively to improve performance.

The federal government can enhance the value of SIBs as a “learning what works” tool by:

  • Stating clearly that the goal is to build reliable evidence about what works, and supporting the evaluation costs of SIB projects but only when they use rigorous research designs capable of reliably attributing causality.

  • Signaling its support for a benefit-cost perspective that values not solely government budget savings, but also economic returns to participants and to society as a whole.

  • Stimulating deals by supporting state and local pay-for-success projects in the form of backend payments as specified in the Social Impact Partnership Act, WIOA, and the Social Innovation Fund Classic competitions, but insisting that state and local agencies commit to sustaining programs that work after a SIB deal ends.

  • Insuring transparency by requiring that public agencies using federal funds must release publicly the final results of evaluations, the payment terms of deals, and final benefit/cost calculations, including the lost savings that would result if the government did not continue to support effective initiatives.

In conclusion, the bipartisan commitment to building evidence by Congress has created a new generation of innovation in education and social policy — better programs, more effective policies, more responsible use of taxpayer dollars. But there is more to be done, including set-asides within funding streams to build evidence, continued use of tiered funding structures, and rewards for localities to move more of their formula funding into programming with evidence.

In the end, it’s not enough to create evidence; we have to make sure it is used. We need to build a culture of continuous improvement and accountability, always striving to learn more and to more effectively direct funding at all levels of government to those programs with the strongest evidence.

Thank you for the opportunity to testify today.

Document Details

Publication Type
Testimony
Date
May 2016
Berlin, Gordon. 2016. “Using Evidence as the Driver of Policy Change.” New York: MDRC.