In 1969 Donald Campbell envisioned an “experimenting society” that systematically tried new programs, rigorously evaluated their effectiveness, and made policy decisions based on evidence rather than ideology.^[1] Campbell’s vision was revolutionary at the time, a commitment to learning what works through continual testing and refinement, and is as relevant as ever today. When social programs fail to achieve their intended outcomes, real people suffer while taxpayers fund ineffective approaches that could be improved or replaced.

More than five decades later, the nation has made important progress, but the gap between Campbell’s vision and reality remains substantial. Evidence is often built too slowly, with long waiting periods between major studies and insufficient data capabilities in the field to run the volume of tests needed for rapid policy learning. The State IMPACT (Innovative Models for Policy Acceleration and Collaborative Testing) Collaborative is designed to change this reality.

“High-Frequency Testing” in the Private Sector

There is a striking contrast between how corporations and public agencies approach evidence building. Companies such as Capital One and most large tech companies routinely conduct thousands of rapid experiments annually, creating a model of continual, embedded evaluation.^[2] This high-frequency experimentation produces faster learning cycles and more responsive adaptation than occasional, large studies.

The corporate sector’s advantage stems largely from sophisticated data infrastructure that seamlessly connects inputs, interventions, and outcomes; dedicated teams with specialized training in experimental design and analysis; and the ability to test relatively straightforward questions with clear, measurable outcomes. For example, when a bank tests a new credit card offer, it has immediate access to the customer’s entire journey, from initial marketing, through application responses, to revenue generation. This combination of connected data architecture, dedicated capabilities, and simple questions about causation enables companies to test at the speed and volume needed for rapid evolution in business practices.

The Contrast with Public Policy Evaluation

While there has been some high-frequency testing in the field of public policy, it is still far from reaching the model of continual, embedded evaluation. What Jim Manzi calls the "causal density" of policy problems—the complex web of factors influencing outcomes—makes high-frequency testing and learning essential. Yet there are many barriers to undertaking it.^[3] For one, the problems public policy aims to solve are entrenched and complex, and it often takes a long time to observe meaningful outcomes. However, complex interventions can be broken into smaller, testable components—different outreach strategies, service-delivery approaches, eligibility criteria, or program features. Public agencies could focus on nearer-term outcomes for these component elements (for example, program enrollment or completion) and draw on existing research that shows how short-term measures predict long-term impacts.^[4]

A bigger issue is that unlike corporations, public-sector agencies often lack integrated data systems. Program input data (for example, data on enrollment and participation) are often not integrated with data on people’s outcomes. It can take months or even years to secure permissions and integrate these data elements. Moreover, even when agencies have good data, they often lack the dedicated research capabilities and methodological expertise to design and analyze high-frequency experiments, the kind of specialized skills that corporations have in-house but that most state agencies must obtain through partnerships. Without this foundation of integrated data and research expertise, it is nearly impossible for agencies to conduct high-frequency testing.

Another issue is a lack of incentives for data sharing and collaboration. Researchers frequently extract data from public agencies to produce academic papers but often fail to build lasting partnerships or return value to those who provide the data. This transactional model discourages data integration and discourages researchers and agencies from working together to build agencies’ data capabilities, both of which are essential for high-frequency testing.  To foster a culture of continual learning, researchers and public agencies need not only better technical systems, but also stronger incentives and norms that support sustained, reciprocal engagement between them.

The State IMPACT Collaborative, led by MDRC and the Coleridge initiative, is designed to address these barriers. Rather than one-off studies, this approach builds sustained relationships that support the volume and continuity of testing needed for systematic policy learning. This work also aligns with objectives of the Evidence Act and Federal Data Strategy by enhancing governments’ ability to use data to make decisions.^[5]

Why State Partnerships Matter

The State IMPACT Collaborative’s focuses on state agencies because they sit at a critical nexus in the public policy ecosystem. They implement federal programs, design state initiatives, and work closely with local communities. They possess deep knowledge of their populations, exist in unique policy contexts, and face pressing challenges. However, many lack the time, technical capabilities, methodological expertise, or data infrastructure to build and use evidence systematically, on a large scale. If they can overcome these challenges, states can become more effective contributors to national policy conversations. Their on-the-ground experiences and insights can inform federal program design and implementation.

Take, as an example, MDRC’s partnership with the Washington Student Achievement Council (which informed the design of the Collaborative). What began as a question about using a chatbot to improve the number of students who completed the Free Application for Federal Student Aid has organically expanded into efforts evaluating the nation’s most generous college scholarship program and studying pathways from high school to postsecondary education and the labor market. This history demonstrates how sustained state partnerships can evolve from isolated studies into comprehensive, evidence-building systems that serve citizens better.

Looking Forward: Building the Experimenting Society

To realize Campbell’s vision of an experimenting society, the United States needs to increase its pace and volume of policy learning dramatically. The State IMPACT Collaborative represents one path toward that goal—a strategy for building distributed capabilities for evidence generation and use across the nation. If it succeeds, it can help create the conditions for more informed policy decisions and a more connected ecosystem where insights can flow across jurisdictional boundaries. And by partnering directly with state agencies—the organizations closest to implementation—it increases the likelihood that evidence will translate into meaningful action. This work is not just about generating more studies, but about creating feedback loops that allow programs to adapt and improve in real time while they serve individuals and families. The goal is to move closer to Campbell's vision of a society that systematically tests, learns, and adapts its approaches based on reliable evidence.

The ideas and conclusions presented herein are the work of the authors listed. Artificial intelligence (AI) tools were used to assist with editing, fact-checking source attributions, research verification, and organization and clarity.