Learning from the Experiments that Never Happened : Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa

Matching grants are one of the most common policy instruments used by developing country governments to try to foster technological upgrading, innovation, exports, use of business development services and other activities leading to firm growth. However, since they involve subsidizing firms, the ris...

Full description

Bibliographic Details
Main Authors:	Campos, Francisco, Coville, Aidan, Fernandes, Ana M., Goldstein, Markus, McKenzie, David
Format:	Journal Article
Language:	en_US
Published:	Elsevier 2014
Subjects:	matching grants impact evaluation randomization bias learning from failure
Online Access:	http://hdl.handle.net/10986/18191

id	okr-10986-18191
recordtype	oai_dc
spelling	okr-10986-181912021-04-23T14:03:42Z Learning from the Experiments that Never Happened : Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa Campos, Francisco Coville, Aidan Fernandes, Ana M. Goldstein, Markus McKenzie, David matching grants impact evaluation randomization bias learning from failure Matching grants are one of the most common policy instruments used by developing country governments to try to foster technological upgrading, innovation, exports, use of business development services and other activities leading to firm growth. However, since they involve subsidizing firms, the risk is that they could crowd out private investment, subsidizing activities that firms were planning to undertake anyway, or lead to pure private gains, rather than generating the public gains that justify government intervention. As a result, rigorous evaluation of the effects of such programs is important. We attempted to implement randomized experiments to evaluate the impact of seven matching grant programs offered in six African countries, but in each case we were unable to complete an experimental evaluation. One critique of development research is publication bias, whereby only “interesting” results get published. Our hope is to mitigate this bias by learning from the experiments that never happened. We describe the three main proximate reasons for lack of implementation: continued project delays, politicians not willing to allow random assignment, and low program take-up; and then delve into the underlying causes of these occurring. Political economy, overly stringent eligibility criteria that do not take account of where value-added may be highest, a lack of attention to detail in “last mile” issues, incentives facing project implementation staff, and the way impact evaluations are funded, and all help explain the failure of randomization. We draw lessons from these experiences for both the implementation and the possible evaluation of future projects. 2014-05-06T19:32:00Z 2014-05-06T19:32:00Z 2014-01-08 Journal Article Journal of the Japanese and International Economies 0889-1583 10.1016/j.jjie.2013.12.007 http://hdl.handle.net/10986/18191 en_US CC By-NC-ND 3.0 IGO http://creativecommons.org/licenses/by-nc-nd/3.0/igo World Bank Elsevier Publications & Research :: Journal Article
repository_type	Digital Repository
institution_category	Foreign Institution
institution	Digital Repositories
building	World Bank Open Knowledge Repository
collection	World Bank
language	en_US
topic	matching grants impact evaluation randomization bias learning from failure
spellingShingle	matching grants impact evaluation randomization bias learning from failure Campos, Francisco Coville, Aidan Fernandes, Ana M. Goldstein, Markus McKenzie, David Learning from the Experiments that Never Happened : Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa
description	Matching grants are one of the most common policy instruments used by developing country governments to try to foster technological upgrading, innovation, exports, use of business development services and other activities leading to firm growth. However, since they involve subsidizing firms, the risk is that they could crowd out private investment, subsidizing activities that firms were planning to undertake anyway, or lead to pure private gains, rather than generating the public gains that justify government intervention. As a result, rigorous evaluation of the effects of such programs is important. We attempted to implement randomized experiments to evaluate the impact of seven matching grant programs offered in six African countries, but in each case we were unable to complete an experimental evaluation. One critique of development research is publication bias, whereby only “interesting” results get published. Our hope is to mitigate this bias by learning from the experiments that never happened. We describe the three main proximate reasons for lack of implementation: continued project delays, politicians not willing to allow random assignment, and low program take-up; and then delve into the underlying causes of these occurring. Political economy, overly stringent eligibility criteria that do not take account of where value-added may be highest, a lack of attention to detail in “last mile” issues, incentives facing project implementation staff, and the way impact evaluations are funded, and all help explain the failure of randomization. We draw lessons from these experiences for both the implementation and the possible evaluation of future projects.
format	Journal Article
author	Campos, Francisco Coville, Aidan Fernandes, Ana M. Goldstein, Markus McKenzie, David
author_facet	Campos, Francisco Coville, Aidan Fernandes, Ana M. Goldstein, Markus McKenzie, David
author_sort	Campos, Francisco
title	Learning from the Experiments that Never Happened : Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa
title_short	Learning from the Experiments that Never Happened : Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa
title_full	Learning from the Experiments that Never Happened : Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa
title_fullStr	Learning from the Experiments that Never Happened : Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa
title_full_unstemmed	Learning from the Experiments that Never Happened : Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa
title_sort	learning from the experiments that never happened : lessons from trying to conduct randomized evaluations of matching grant programs in africa
publisher	Elsevier
publishDate	2014
url	http://hdl.handle.net/10986/18191
_version_	1764439420233056256

Learning from the Experiments that Never Happened : Lessons from Trying to Conduct Randomized Evaluations of Matching Grant Programs in Africa

Similar Items