What Can Schools, Colleges, and Youth Programs Do With Predictive Analytics?
THE PROBLEM: HOW TO ASSESS YOUNG PEOPLE’S RISK USING MORE DATA BUT LESS TIME
Many low-income young people are not reaching important milestones for success (for example, completing a program or graduating from school on time). But the social-service organizations and schools that serve them often struggle to identify who is at more or less risk. These institutions often either over- or underestimate risk, missing opportunities to intervene with those who need more help or inefficiently providing services to those who do not need them. Most “early warning systems” of risk rely on only a few measures, mainly because in the course of day-to-day practice, one can keep track of only so much information at a time. Yet this approach ignores a wealth of data collected for different purposes that could help programs and schools identify risk earlier and more accurately.
A school administrator sees students struggling to pass the ninth-grade algebra exam and wants to know each student’s risk of failing. She has access to students’ past course histories and math assignments. She also thinks more factors may be at work, but reviewing more data feels overwhelming.
A SOLUTION: PREDICTIVE ANALYTICS
MDRC uses cutting-edge methods and a field-tested framework to capitalize on that wealth of data. With predictive analytics, we can:
- Rank young people by their risk levels
- Show variation in young people’s risks and needs at a single point in time
- Capture changes in risk as new information is collected
To get a better sense of who is at risk of failing algebra, the administrator wants to take advantage of past course history and performance on math assignments, as well as factors such as whether a student is an English language learner, whether that student has moved frequently, and the student’s attendance patterns. Since she does not have time or know how to combine and analyze those data, she needs an approach that can help while producing an easy-to-interpret result.
THE RESULT: REAL-TIME ESTIMATES OF INDIVIDUALS’ RISKS THAT CAPTURE VARIATION
MDRC can apply predictive analytics to identify risk levels for individual young people but also aggregate risk levels for sites (schools or program locations), so that institutions can better direct their resources.
Predictive analytics can answer questions such as:
- Absence: What is the likelihood that an individual will miss school or program participation more than 10 percent of the year?
- Course failure: Which students are most likely to fail a required exam in a given subject?
- Completion: Is a young person more at risk of not completing Program A or Program B?
- Variation in risk: How is the risk of not graduating on time spread among schools or centers in a district?
- Change in risk: How do young people’s risks of not completing a program change during the course of a year?
The administrator wants to see whether students who are not at risk of failing at the beginning of the ninth grade become at risk of failing six weeks into the school year. This analysis can help her understand who might need additional services and when.
WHAT IT TAKES
- MDRC works with an organization to assess whether predictive analytics are appropriate and valuable for its goals and questions.
Are predictive analytics right for your organization? Yes, if you:
- Already assess or guess at young people’s risk of not reaching certain milestones, and want to improve the accuracy of your risk assessments.
- Can implement interventions meant to help young people at risk.
- Make the most of limited resources by directing services only to those at risk.
- Believe that targeting services based on risk will increase the success of an intervention.
- If so, we investigate how ready the organization is for predictive analytics.
Is your organization ready for predictive analytics? Yes, if you:
- Make data readily available that are updated throughout the year.
- Are ready to communicate and act on results.
- Employ a staff and a data system that can sustain predictive work.
- MDRC works with the organization to identify which milestones we should focus on. We choose ones where there are enough good data to work with and where the risk involved is relevant to the organization’s decisions about interventions.
- For each milestone we choose with the organization, we calculate a likelihood that each individual will not reach that milestone.
- We help the organization incorporate these results into its tracking systems. We help it prepare to sustain predictive analytics and incorporate the approach into its continuous improvement processes.
We also help organizations understand the limitations of predictive analytics. Predictive analytics can only assess risk — they do not reveal anything about the effectiveness of an intervention. But MDRC can use other rigorous methods, such as randomized field trials, to answer those questions. Our experience from evaluating dozens of youth programs as well as designing interventions informed by behavioral science can help organizations decide what to do next: how best to intervene and with whom.
For more information on predictive analytics, e-mail Rekha Balu.
Document Details
Publication Type
Brief
Date
April 2017
Balu, Rekha and Kristin Porter. 2017. “What Can Schools, Colleges, and Youth Programs Do With Predictive Analytics?.” New York: MDRC.
