Issue Focus

Guiding Questions for Supporting Culturally Responsive Evaluation Practices and an Equity-Based Perspective

| Oscar Cerna, Barbara Condliffe, Ashley Wilson

The credibility of MDRC’s work depends on providing both rigorous and relevant research. It also depends on recognizing that multiple perspectives and frameworks are necessary for better understanding the cultural priorities and needs of the populations we study. At MDRC, this leads us to conclude that adopting more culturally responsive research practices and an equity-based perspective is essential for maximizing the benefits of our work. MDRC’s Equity Collaborative has been learning about culturally responsive evaluation practices and equity-based approaches while considering the challenges and opportunities of integrating these practices into more of our work. In this document, we offer definitions for culturally responsive evaluation and an equity-based perspective, then describe how MDRC staff and the broader policy evaluation field can apply these practices and perspectives.[1]

Over the last two decades, the fields of applied research and evaluation have begun defining and characterizing culturally responsive approaches.[2] For instance, culturally responsive evaluation (CRE) “recognizes that demographic, sociopolitical, and contextual dimensions, locations, perspectives, and characteristics of culture matter fundamentally in evaluation.”[3] It “gives particular attention to groups that have been historically marginalized.”[4] Being culturally responsive also requires reflecting on one’s own culture and relationship to the culture(s) within which evaluators operate.

CRE asks that evaluators broaden the scope of stakeholders typically involved in a research project and ensure that the design and implementation of studies reflect the voices of all relevant stakeholders. CRE also requires that evaluators be mindful of inherent power dynamics between researchers and research subjects that may inadvertently affect the data collection, definition of outcomes, interpretation of results, and reporting priorities. It encourages evaluators to seek alternative ways to collect data that may lessen these dynamics, build trust in the process with research subjects, and facilitate more honest responses. Part of the aim of CRE is for evaluators to redefine how they approach research with communities by both developing more inclusive research practices and by making efforts to shift power dynamics between researchers and research subjects.[5]

While CRE addresses how we engage in policy-relevant research, an equity-based perspective challenges us to examine what projects we pursue and what research questions we address. An agenda of evidence-building for equity might include implementation studies that describe the current conditions of equity/inequity, impact evaluations that measure whether interventions reduce inequities between groups, and demonstrations that design and test interventions to address inequities.

Given the importance of increased equity across educational and social service systems, MDRC will be increasingly drawing on CRE and equity-based perspectives across the research and technical assistance we conduct. This will enhance the work we undertake and its responsiveness to the communities and populations who participate in our studies, increasing the odds that they will genuinely benefit from our work.

For each phase of a hypothetical evaluation or study, the table in the full document includes a framing statement about including practices of cultural responsiveness and equity and key questions to consider for integrating these practices. We hope that our colleagues find this resource useful for considering how to apply an equity lens and culturally responsive practices in different types of education and social policy research projects, including implementation studies, impact evaluations, demonstrations, technical assistance, and continuous improvement efforts.

[1] We consulted resources, including Hood, Hopson, and Kirkhart (2015); Hopson (2009); Public Policy Associates, Inc. (2015); and Johnson and Anderson (2019).

[2] Hood et al. (2015).

[3] Hopson (2009).

[4] Hood et al. (2015).

[5] Table 12.1 of Hood et al. (2015) provides an overview of the key characteristics of CRE.