Filling All the Seats in the Room
Using Data to Analyze Enrollment Drop-Off
Meeting enrollment goals is as much a function of recruiting eligible participants as it is a function of helping them successfully complete the enrollment process. When recruiting to fill spots in a program, staff members focus on finding the right match between program services and requirements, and an applicant’s needs and interests. A match is made when needs, interests, and requirements align, and results in a successful enrollment. Many programs, like those in MDRC’s evaluation of the WorkAdvance project, find that just a fraction of those they initially recruit to participate in their program end up enrolling. In this post, we examine some key lessons from MDRC’s evaluation of the WorkAdvance project to help turn program recruits into program success stories.
MDRC’s technical assistance teams developed three steps and a few simple tools to help staff in the WorkAdvance program analyze their recruitment process, with these goals in mind:
Use resources efficiently by recruiting at locations that yield a high number of eligible applicants
Improve their understanding of when and why applicants drop out of the enrollment process in order to address barriers
On average, 1 in 5 people recruited for WorkAdvance were eligible for the program
A sector-focused workforce program such as WorkAdvance had a specific end-goal: Enrolling participants in structured training and employment services that would lead to jobs in a specific industry. Recruiters had to find applicants who met specific eligibility criteria, including whether they had the skills and background that employers sought. Recruiters also had to carefully assess whether potential participants were willing and able to attend program services.
Three Steps to More Efficient Enrollment
Step 1: Mine your data
WorkAdvance programs used a Recruitment Questionnaire to gather some basic information, asking a short list of questions early in the enrollment process and recording responses in their Management Information System (MIS) for easy analysis. Here is some of the most important information to collect at this stage:
How did the applicant learn about the program? Be as specific as possible. WorkAdvance programs tested out numerous outreach strategies for connecting with potential applicants. Recording this information in detail helps staff understand which strategies work best.
Did the applicant enroll in the program?
If not, why? Be as specific as possible. This helps staff understand potential barriers to enrollment.
Here is an example of a well-executed recruitment questionnaire:
Step 2: Target promising recruitment sources
MDRC technical liaisons used data from the recruitment questionnaire to create a referral sources matrix, which staff used to assess which recruitment strategies and sources brought in the most enrollees. In the sample matrix below, two takeaways stand out:
These data may suggest that more complete information about the program’s services and requirements could be communicated via online sources; and that resources devoted to some sources like newspapers, flyers, TV/radio, and direct outreach to employers may be put to better use elsewhere.
Programs can use a recruitment questionnaire to determine the reasons applicants may not enroll. Programs may discover, for example, that enrollees have trouble accessing their offices, or that their office hours make it difficult for working parents to attend. In response, staff and management can consider strategies like offering bus tickets or scheduling extended hours on certain days of the week.
Step 3: Analyze where drop-off happens
WorkAdvance programs used a third tool, the recruitment funnel, to understand where participants were most likely to drop out in the recruitment and enrollment process. Did applicants not meet the enrollment criteria? Did they meet the criteria, but come to the initial enrollment appointment without the necessary paperwork? Did they complete the paperwork, but neglect to return for the orientation meeting? Let’s examine what the program learned in this example funnel:
A recruitment funnel provides clear data and an important opportunity for program administrators to critically examine what’s working in their processes and make changes.
There are numerous options for addressing challenges at each drop-off point: Programs can test different solutions that fit their program context, reevaluate the funnel, and adjust strategies based on new data.
MDRC’s Center for Applied Behavioral Sciences (CABS) SIMPLER framework can provide guidance in diagnosing problems at each step of the funnel and help to brainstorm possible solutions. SIMPLER stands for Social influence, Implementation prompts, Make deadlines, Personalization, Loss aversion, Ease, and Reminders—seven practical behavioral techniques that can be applied to service delivery challenges in a variety of ways. For example:
If drop-off occurs because applicants come to the enrollment meeting without the right documentation:
Flyers/outreach materials may need to be simplified so that they clearly state what documents are needed
Recruiters can ask partner organizations to communicate these requirements clearly when making a referral
Reception staff who schedule enrollment appointments could work from a written script to outline all the necessary paperwork
If drop-off occurs when applicants do not return for the orientation:
This may suggest that recruiters need to send reminder texts
Recruiters may need to make contact after the missed appointment to understand a participant’s difficulty in attending meetings
Programs may consider offering bus or metro tickets, or any other supports that can address barriers to attendance
Programs may consider offering the orientation the same day as the enrollment appointment