Adaptive Experiments for Policy Choice : Phone Calls for Home Reading in Kenya

Adaptive sampling in experiments with multiple waves can improve learning for “policy choice problems” where the goal is to select the optimal intervention or treatment among several options. This paper uses a real-world policy choice problem to de...

Full description

Bibliographic Details
Main Authors:	Esposito, Bruno, Sautmann, Anja
Format:	Working Paper
Language:	English
Published:	World Bank, Washington, DC 2022
Subjects:	ADAPTIVE EXPERIMENTS AUTOMATED CALLS TO PARENTS MULTI-ARMED BANDITS EDUCATION TECHNOLOGY EARLY LITERACY ADAPTIVE SAMPLING RESEARCH DESIGN PARAMETERS ECONOMIC DEVELOPMENT RESEARCH ECONOMIC POLICY RESEARCH METHODS EDTECH POLICY CHOICES
Online Access:	http://documents.worldbank.org/curated/en/099337506232226851/IDU078c6dff10259d0409d0997f0d753a7e2cb72 http://hdl.handle.net/10986/37602

id	okr-10986-37602
recordtype	oai_dc
spelling	okr-10986-376022022-06-29T05:11:02Z Adaptive Experiments for Policy Choice : Phone Calls for Home Reading in Kenya Esposito, Bruno Sautmann, Anja ADAPTIVE EXPERIMENTS AUTOMATED CALLS TO PARENTS MULTI-ARMED BANDITS EDUCATION TECHNOLOGY EARLY LITERACY ADAPTIVE SAMPLING RESEARCH DESIGN PARAMETERS ECONOMIC DEVELOPMENT RESEARCH ECONOMIC POLICY RESEARCH METHODS EDTECH POLICY CHOICES Adaptive sampling in experiments with multiple waves can improve learning for “policy choice problems” where the goal is to select the optimal intervention or treatment among several options. This paper uses a real-world policy choice problem to demonstrate the advantages of adaptive sampling and propose solutions to common issues in applying the method. The application is a test of six formats for automated calls to parents in Kenya that encourage reading with children at home. The adaptive ‘exploration sampling’ algorithm is used to efficiently identify the call with the highest rate of engagement. Simulations show that adaptive sampling increased the posterior probability of the chosen arm being optimal from 86 to 93 percent and more than halved the posterior expected regret. The paper discusses a range of implementation aspects, including how to decide about research design parameters such as the number of experimental waves. 2022-06-28T15:42:33Z 2022-06-28T15:42:33Z 2022-06 Working Paper http://documents.worldbank.org/curated/en/099337506232226851/IDU078c6dff10259d0409d0997f0d753a7e2cb72 http://hdl.handle.net/10986/37602 English Policy Research Working Paper;10098 CC BY 3.0 IGO http://creativecommons.org/licenses/by/3.0/igo World Bank World Bank, Washington, DC Policy Research Working Paper Publications & Research Kenya
repository_type	Digital Repository
institution_category	Foreign Institution
institution	Digital Repositories
building	World Bank Open Knowledge Repository
collection	World Bank
language	English
topic	ADAPTIVE EXPERIMENTS AUTOMATED CALLS TO PARENTS MULTI-ARMED BANDITS EDUCATION TECHNOLOGY EARLY LITERACY ADAPTIVE SAMPLING RESEARCH DESIGN PARAMETERS ECONOMIC DEVELOPMENT RESEARCH ECONOMIC POLICY RESEARCH METHODS EDTECH POLICY CHOICES
spellingShingle	ADAPTIVE EXPERIMENTS AUTOMATED CALLS TO PARENTS MULTI-ARMED BANDITS EDUCATION TECHNOLOGY EARLY LITERACY ADAPTIVE SAMPLING RESEARCH DESIGN PARAMETERS ECONOMIC DEVELOPMENT RESEARCH ECONOMIC POLICY RESEARCH METHODS EDTECH POLICY CHOICES Esposito, Bruno Sautmann, Anja Adaptive Experiments for Policy Choice : Phone Calls for Home Reading in Kenya
geographic_facet	Kenya
relation	Policy Research Working Paper;10098
description	Adaptive sampling in experiments with multiple waves can improve learning for “policy choice problems” where the goal is to select the optimal intervention or treatment among several options. This paper uses a real-world policy choice problem to demonstrate the advantages of adaptive sampling and propose solutions to common issues in applying the method. The application is a test of six formats for automated calls to parents in Kenya that encourage reading with children at home. The adaptive ‘exploration sampling’ algorithm is used to efficiently identify the call with the highest rate of engagement. Simulations show that adaptive sampling increased the posterior probability of the chosen arm being optimal from 86 to 93 percent and more than halved the posterior expected regret. The paper discusses a range of implementation aspects, including how to decide about research design parameters such as the number of experimental waves.
format	Working Paper
author	Esposito, Bruno Sautmann, Anja
author_facet	Esposito, Bruno Sautmann, Anja
author_sort	Esposito, Bruno
title	Adaptive Experiments for Policy Choice : Phone Calls for Home Reading in Kenya
title_short	Adaptive Experiments for Policy Choice : Phone Calls for Home Reading in Kenya
title_full	Adaptive Experiments for Policy Choice : Phone Calls for Home Reading in Kenya
title_fullStr	Adaptive Experiments for Policy Choice : Phone Calls for Home Reading in Kenya
title_full_unstemmed	Adaptive Experiments for Policy Choice : Phone Calls for Home Reading in Kenya
title_sort	adaptive experiments for policy choice : phone calls for home reading in kenya
publisher	World Bank, Washington, DC
publishDate	2022
url	http://documents.worldbank.org/curated/en/099337506232226851/IDU078c6dff10259d0409d0997f0d753a7e2cb72 http://hdl.handle.net/10986/37602
_version_	1764487519963971584

Adaptive Experiments for Policy Choice : Phone Calls for Home Reading in Kenya

Similar Items