On the Implications of Essential Heterogeneity for Estimating Causal Impacts Using Social Experiments

On the Implications of Essential Heterogeneity for Estimating Causal Impacts Using Social Experiments

Randomized control trials are sometimes used to estimate the aggregate benefit from some policy or program. To address the potential bias from selective take-up, the randomization is used as an instrumental variable for treatment status. Does this...

Full description

Bibliographic Details
Main Author: Ravallion, Martin
Format: Policy Research Working Paper
Language:English
Published: 2012
Subjects:
COUNTERFACTUAL
DEVELOPMENT RESEARCH
DISEASE
ECONOMETRICS
ESTIMATORS
IMPACT EVALUATION
INCOME
INSTRUMENTAL VARIABLES
INTERVENTION
LEARNING
MODELING
PROGRAMS
RESEARCH WORKING PAPERS
SOCIAL EXPERIMENTS
SOCIAL PROGRAMS
TARGETING
TRAINING PROGRAMS
TREATMENT
TREATMENT EFFECTS
VARIABILITY
Online Access:http://www-wds.worldbank.org/external/default/main?menuPK=64187510&pagePK=64193027&piPK=64187937&theSitePK=523679&menuPK=64187510&searchMenuPK=64187283&siteName=WDS&entityID=000158349_20110921143338
http://hdl.handle.net/10986/3568
id okr-10986-3568
recordtype oai_dc
spelling okr-10986-35682021-04-23T14:02:10Z On the Implications of Essential Heterogeneity for Estimating Causal Impacts Using Social Experiments Ravallion, Martin COUNTERFACTUAL DEVELOPMENT RESEARCH DISEASE ECONOMETRICS ESTIMATORS IMPACT EVALUATION INCOME INSTRUMENTAL VARIABLES INTERVENTION LEARNING MODELING PROGRAMS RESEARCH WORKING PAPERS SOCIAL EXPERIMENTS SOCIAL PROGRAMS TARGETING TRAINING PROGRAMS TREATMENT TREATMENT EFFECTS VARIABILITY Randomized control trials are sometimes used to estimate the aggregate benefit from some policy or program. To address the potential bias from selective take-up, the randomization is used as an instrumental variable for treatment status. Does this (popular) method of impact evaluation help reduce the bias when take-up depends on unobserved gains from take up? Such "essential heterogeneity" is known to invalidate the instrumental variable estimator of mean causal impact, though one still obtains another parameter of interest, namely mean impact amongst those treated. However, if essential heterogeneity is the only problem then the naïve (ordinary least squares) estimator also delivers this parameter; there is no gain from using randomization as an instrumental variable. On allowing the heterogeneity to also alter counterfactual outcomes, the instrumental variable estimator may well be more biased for mean impact than the naïve estimator. Examples are given for various stylized programs, including a training program that attenuates the gains from higher latent ability, an insurance program that compensates for losses from unobserved risky behavior and a microcredit scheme that attenuates the gains from access to other sources of credit. Practitioners need to think carefully about the likely behavioral responses to social experiments in each context. 2012-03-19T18:04:44Z 2012-03-19T18:04:44Z 2011-09-01 http://www-wds.worldbank.org/external/default/main?menuPK=64187510&pagePK=64193027&piPK=64187937&theSitePK=523679&menuPK=64187510&searchMenuPK=64187283&siteName=WDS&entityID=000158349_20110921143338 http://hdl.handle.net/10986/3568 English Policy Research working paper ; no. WPS 5804 CC BY 3.0 IGO http://creativecommons.org/licenses/by/3.0/igo/ World Bank Publications & Research :: Policy Research Working Paper The World Region The World Region
repository_type Digital Repository
institution_category Foreign Institution
institution Digital Repositories
building World Bank Open Knowledge Repository
collection World Bank
language English
topic COUNTERFACTUAL
DEVELOPMENT RESEARCH
DISEASE
ECONOMETRICS
ESTIMATORS
IMPACT EVALUATION
INCOME
INSTRUMENTAL VARIABLES
INTERVENTION
LEARNING
MODELING
PROGRAMS
RESEARCH WORKING PAPERS
SOCIAL EXPERIMENTS
SOCIAL PROGRAMS
TARGETING
TRAINING PROGRAMS
TREATMENT
TREATMENT EFFECTS
VARIABILITY
spellingShingle COUNTERFACTUAL
DEVELOPMENT RESEARCH
DISEASE
ECONOMETRICS
ESTIMATORS
IMPACT EVALUATION
INCOME
INSTRUMENTAL VARIABLES
INTERVENTION
LEARNING
MODELING
PROGRAMS
RESEARCH WORKING PAPERS
SOCIAL EXPERIMENTS
SOCIAL PROGRAMS
TARGETING
TRAINING PROGRAMS
TREATMENT
TREATMENT EFFECTS
VARIABILITY
Ravallion, Martin
On the Implications of Essential Heterogeneity for Estimating Causal Impacts Using Social Experiments
geographic_facet The World Region
The World Region
relation Policy Research working paper ; no. WPS 5804
description Randomized control trials are sometimes used to estimate the aggregate benefit from some policy or program. To address the potential bias from selective take-up, the randomization is used as an instrumental variable for treatment status. Does this (popular) method of impact evaluation help reduce the bias when take-up depends on unobserved gains from take up? Such "essential heterogeneity" is known to invalidate the instrumental variable estimator of mean causal impact, though one still obtains another parameter of interest, namely mean impact amongst those treated. However, if essential heterogeneity is the only problem then the naïve (ordinary least squares) estimator also delivers this parameter; there is no gain from using randomization as an instrumental variable. On allowing the heterogeneity to also alter counterfactual outcomes, the instrumental variable estimator may well be more biased for mean impact than the naïve estimator. Examples are given for various stylized programs, including a training program that attenuates the gains from higher latent ability, an insurance program that compensates for losses from unobserved risky behavior and a microcredit scheme that attenuates the gains from access to other sources of credit. Practitioners need to think carefully about the likely behavioral responses to social experiments in each context.
format Publications & Research :: Policy Research Working Paper
author Ravallion, Martin
author_facet Ravallion, Martin
author_sort Ravallion, Martin
title On the Implications of Essential Heterogeneity for Estimating Causal Impacts Using Social Experiments
title_short On the Implications of Essential Heterogeneity for Estimating Causal Impacts Using Social Experiments
title_full On the Implications of Essential Heterogeneity for Estimating Causal Impacts Using Social Experiments
title_fullStr On the Implications of Essential Heterogeneity for Estimating Causal Impacts Using Social Experiments
title_full_unstemmed On the Implications of Essential Heterogeneity for Estimating Causal Impacts Using Social Experiments
title_sort on the implications of essential heterogeneity for estimating causal impacts using social experiments
publishDate 2012
url http://www-wds.worldbank.org/external/default/main?menuPK=64187510&pagePK=64193027&piPK=64187937&theSitePK=523679&menuPK=64187510&searchMenuPK=64187283&siteName=WDS&entityID=000158349_20110921143338
http://hdl.handle.net/10986/3568
_version_ 1764387242331078656

Similar Items