At the design stage you will define your evaluation questions, identify an appropriate methodology, and plan and budget for the evaluation activities. Use the following guidelines to help design and plan your impact evaluation.
Impact evaluations assess the changes in development outcomes that are caused by a particular project, program, or policy. To establish a causal relationship, impact evaluations rely on a set of experimental and quasi-experimental methods. The following links offer an overview of the main methodologies:
a. Introductory readings:
- Impact evaluation in practice (Gertler et al.; World Bank, 2010).
- Evaluating the impact of development projects on poverty. A handbook for practitioners(Baker, Judy; World Bank, 2000).
b. Intermediate readings:
- Evaluating anti-poverty programs (Ravallion, Martin; World Bank, 2005).
- Program evaluation and spillover effects (Angelucci and Di Maro; IDB, 2010).
- Handbook on impact evaluation: quantitative methods and practices (Khandker et al.; World Bank, 2009).
- Recent developments in the econometrics of impact evaluation (Imbens and Wooldrige; Journal of Economic Literature, 2009).
- The mystery of vanishing benefits: an introduction to impact evaluation (Ravallion, Martin; World Bank Economic Review, Vol. 15, No. 1).
a. Experimental methods:
- In pursuit of balance. Randomization in practice in development field experiments (Bruhn and McKenzie; World Bank, 2008).
- Using randomization in development economics research:a toolkit (Duflo et al.; Center for Economic Policy Research, 2007).
b. Quasi-experimental methods:
- A primer for applying propensity-score matching (Heinrich et al; IDB, 2010).
- Instrumental variables and the search for ddentification: from supply and demand to natural experiment (Angrist and Krueger; Journal of Economic Perspectives, 2001).
- Regression discontinuity designs:a guide to practice (Imbens and Lemieux; National Bureau of Economic Research, 2007).
Designing impact evaluations for agricultural projects (Winters et al.; IDB, 2010).
Early childhood development
Methodologies to evaluate early childhood development programs (Behrman et al.; World Bank, 2007).
Impact evaluation for school-based management reform (Gertler et al.; World Bank, 2007).
Guidelines for impact evaluation in education using experimental design (Bando, Rosangela; IDB, 2013).
Impact evaluation for slum upgrading interventions (Field and Kremer; World Bank, 2008).
Evaluating the impact of cluster development programs (Giuliani et al.; IDB,2013).
Building in an evaluation component for active labor market programs: a practitioner’s guide(Card et al.; IDB, 2011).
Impact evaluation for land property rights reforms (Conning and Deb; World Bank, 2007).
Impact evaluation for microfinance (Karlan and Goldberg; World Bank, 2007).
Methodologies to evaluate the impact of large scale nutrition programs (Habicht et al.; World Bank, 2009).
Evaluating the impact of regional development programs (Winters and Sitja; IDB, 2010).
Science and technology
Evaluating the impact of science, technology and innovation programs:a methodological toolkit(Crespi et al.; IDB, 2011).
Technical guidelines for evaluating the impacts of tourism using simulation models (Taylor, Edward; IDB, 2010).
Conducting impact evaluations in urban transport (Boarnet, Marlon; World Bank, 2007).
Impact evaluation of rural roads projects (Van de Walle, Dominique; World Bank, 2008).
Water and sanitation
A guide to water and sanitation sector impact evaluations (Poulos et al.; World Bank, 2006).
- Power calculations spreadsheet: a tool for determining sample sizes.
- Impact evaluation Gantt chart: a spreadsheet to assist in the scheduling of impact evaluation activities.
- Data collection Gantt chart: a spreadsheet to assist in the scheduling of data collection activities.
- Budget template: a tool for estimating the costs of an impact evaluation.
- Concept note template: a document that describes the details of the impact evaluation methodology.
- Design template: a blank presentation to guide the main steps and components of an impact evaluation design.
- Impact evaluation checklist: a list of core activities to be carried out during an impact evaluation.