External Validity and Policy Adaptation : From Impact Evaluation to Policy Design

With the growing number of impact evaluations worldwide, the question of how to apply this evidence in policy making processes has arguably become the main challenge for evidence-based policy making. How can policy makers predict whether a policy will have the same impact in their context as it did...

Full description

Bibliographic Details
Main Author:	Williams, Martin J.
Format:	Journal Article
Published:	Published by Oxford University Press on behalf of the World Bank 2022
Subjects:	IMPACT EVALUATION EXTERNAL VALIDITY POLICY DESIGN EVIDENCE-BASED POLICY MONITORING AND EVALUATION PROJECT ANALYSIS
Online Access:	http://hdl.handle.net/10986/36809

id	okr-10986-36809
recordtype	oai_dc
spelling	okr-10986-368092022-01-12T05:10:39Z External Validity and Policy Adaptation : From Impact Evaluation to Policy Design Williams, Martin J. IMPACT EVALUATION EXTERNAL VALIDITY POLICY DESIGN EVIDENCE-BASED POLICY MONITORING AND EVALUATION PROJECT ANALYSIS With the growing number of impact evaluations worldwide, the question of how to apply this evidence in policy making processes has arguably become the main challenge for evidence-based policy making. How can policy makers predict whether a policy will have the same impact in their context as it did elsewhere, and how should this influence the policy’s design and implementation? This paper suggests that failures of external validity (both in transporting and scaling up policy) can be understood as arising from an interaction between a policy’s theory of change and a dimension of the context in which it is being implemented. The paper surveys existing approaches to analyzing external validity, and suggests that there has been more focus on the generalizability of impact evaluation results than on the applicability of evidence to specific contexts. To help fill this gap, the study develops a method of “mechanism mapping” that maps a policy’s theory of change against salient contextual assumptions to identify external validity problems and suggest appropriate policy adaptations. In deciding whether and how to adapt a policy, there is a fundamental informational trade-off between the strength of evidence on the policy from other contexts and the policy maker’s information about the local context. 2022-01-11T18:06:11Z 2022-01-11T18:06:11Z 2020-07 Journal Article World Bank Research Observer 1564-6971 http://hdl.handle.net/10986/36809 CC BY-NC-ND 3.0 IGO http://creativecommons.org/licenses/by-nc-nd/3.0/igo World Bank Published by Oxford University Press on behalf of the World Bank Publications & Research Publications & Research :: Journal Article Africa Africa Eastern and Southern (AFE) Zimbabwe
repository_type	Digital Repository
institution_category	Foreign Institution
institution	Digital Repositories
building	World Bank Open Knowledge Repository
collection	World Bank
topic	IMPACT EVALUATION EXTERNAL VALIDITY POLICY DESIGN EVIDENCE-BASED POLICY MONITORING AND EVALUATION PROJECT ANALYSIS
spellingShingle	IMPACT EVALUATION EXTERNAL VALIDITY POLICY DESIGN EVIDENCE-BASED POLICY MONITORING AND EVALUATION PROJECT ANALYSIS Williams, Martin J. External Validity and Policy Adaptation : From Impact Evaluation to Policy Design
geographic_facet	Africa Africa Eastern and Southern (AFE) Zimbabwe
description	With the growing number of impact evaluations worldwide, the question of how to apply this evidence in policy making processes has arguably become the main challenge for evidence-based policy making. How can policy makers predict whether a policy will have the same impact in their context as it did elsewhere, and how should this influence the policy’s design and implementation? This paper suggests that failures of external validity (both in transporting and scaling up policy) can be understood as arising from an interaction between a policy’s theory of change and a dimension of the context in which it is being implemented. The paper surveys existing approaches to analyzing external validity, and suggests that there has been more focus on the generalizability of impact evaluation results than on the applicability of evidence to specific contexts. To help fill this gap, the study develops a method of “mechanism mapping” that maps a policy’s theory of change against salient contextual assumptions to identify external validity problems and suggest appropriate policy adaptations. In deciding whether and how to adapt a policy, there is a fundamental informational trade-off between the strength of evidence on the policy from other contexts and the policy maker’s information about the local context.
format	Journal Article
author	Williams, Martin J.
author_facet	Williams, Martin J.
author_sort	Williams, Martin J.
title	External Validity and Policy Adaptation : From Impact Evaluation to Policy Design
title_short	External Validity and Policy Adaptation : From Impact Evaluation to Policy Design
title_full	External Validity and Policy Adaptation : From Impact Evaluation to Policy Design
title_fullStr	External Validity and Policy Adaptation : From Impact Evaluation to Policy Design
title_full_unstemmed	External Validity and Policy Adaptation : From Impact Evaluation to Policy Design
title_sort	external validity and policy adaptation : from impact evaluation to policy design
publisher	Published by Oxford University Press on behalf of the World Bank
publishDate	2022
url	http://hdl.handle.net/10986/36809
_version_	1764485939324780544

External Validity and Policy Adaptation : From Impact Evaluation to Policy Design

Similar Items