Phase 1: Planning

Forecasting the Size of the Evaluation Sample

When planning a randomized controlled trial (RCT) of a program, policy, or practice, researchers often forecast expected sample sizes. This step is essential for budget planning and ensuring the evaluation is large enough to detect a meaningful effect (that is, it is adequately powered).

One widely used method is a funnel analysis, which relies on historical data from the study college (or colleges) to estimate how many individuals would have been eligible for the program in a past cohort. This analysis helps forecast how many might be eligible in a future cohort.

Before beginning, review this funnel guidance memo, which outlines key considerations for

  • defining intervention eligibility criteria,
  • deciding whom to randomize, and
  • forecasting intervention take-up rates.

When you are ready to request data, this simple data request template provides sample language to help initiate the process with a college or system. If you are new to Excel, this sample funnel analysis spreadsheet demonstrates the basic descriptive analyses involved in a typical funnel analysis.

In addition, MDRC offers two point-and-click funnel tools in Microsoft Excel that enable researchers or college partners to perform a funnel analysis after inserting data into the workbook. These tools require some data preparation, so they are most useful when eligibility criteria are still flexible and researchers want to explore how different criteria affect sample size. One version of the tool contains 17 prespecified and commonly used eligibility characteristics, while the other version allows for additional “flexible variables” to accommodate less common eligibility requirements.

Key Resources

Memo
Forecasting the Size of the Study Sample Using a Funnel Analysis
Offers essential guidance on conducting a funnel analysis

Template
Funnel Data Request Template
Sample data request form to send to staff members at colleges or college systems

Tool
Sample Funnel Analysis Spreadsheet
Excel workbook with sample data and formulas for funnel analyses; ideal for one-time calculations once eligibility criteria are set

Video: Tour the Resources