APPLIED MICRO: Adapting for scale: Experimental evidence on computer-aided instruction in India; Professor Karthik Muralidharan (University of California, San Diego)

Abstract

The promise of ‘evidence-based policy’ for improving human welfare is critically constrained by the growing evidence that interventions found to be effective in small-scale efficacy trials are often ineffective at larger scales. We study the scaling of a personalized adaptive learning (PAL) software that was highly effective in a small-scale efficacy trial. We document how to adapt PAL software for scale, and experimentally evaluate its deployment in a diverse sample 20 times larger than the original study. We find that the scaled intervention continued to be highly effective, with treated students scoring 0.22SD and 0.2SD higher in Mathematics and Hindi after 18 months, an increase in productivity of 50-66\% over business-as-usual (control group) value-added. These gains remained stable after a third year of implementation with reduced staffing. We also show that the time logged by students on the PAL platform strongly correlates with learning gains, which provides a low-cost proxy measure of implementation quality that can be used to predict impact of future scale-ups. These results are directly relevant for realizing the potential of technology to improve the productivity of education systems at scale.