Explore our hub for assessing development projects
The estimation of benefits is what separates CBA and CEA and the analysis presented here requires explicitly noting all of the benefits to be included in the analysis as well as the monetary values of those benefits. All relevant project benefits should be noted. Since a societal perspective is taken in CBA for Bank-funded projects, the benefits should include all societal benefits.
The choice of method will depend on many factors. The critical factors are four:
First
Whether a market exists or does not for the goods and services to be provided by the project. If there is a market, price can be elicited directly from published sources or from direct observation. If not, price information will have to be derived from either revealed or stated preferences.
Second
The nature of the Theory of Change. The logic of the intervention and the identified pathways through which the proposed intervention brings the expected impact are central in identifying the mechanisms by which those impacts can be monetized. A credible transition from outputs to outcomes and impacts will hinge on the quality of the assumptions embedded in the Theory of Change and this sequence and logic needs to be consistent with the step required to monetize impacts.
Third
The quality, availability and timeliness (how recent it is?) of the information/data required for benefit estimation.
Fourth
Sector-specific considerations and standard professional practice. Irrespective of the method, benefit estimation should be clearly linked to the logic chain: input→ output→ outcome→ impact (benefits) chain.
For more information, visit the FAQ at the top of this page.
Cost Benefit Analysis in the Context of the Energy Infrastructure Package (Nils-Henrik von der Fehr - THINK-European Union, 2012)
Cost-Effectiveness Analysis of education and Health Interventions in Developing Countries(McEwan, Patrick, J. -IADB, 2011)
Can Cost-Benefit Analysis Guide Education policy in Developing Countries? (Jimenez, Emmanuel; Patrinos, Harry Anthony - World Bank, 2008)
A Review of the Valuation Of Environmental Costs and Benefits in World Bank (Patricia Silva, Stefano Pagiola- World Bank, 2005)
Cost-benefit Analysis of Natural Disaster Risk Management in Developing Countries(International Finance Corporation, Manual 2005)
There are different approaches in identifying costs including the ingredient method which relies on the identification of all resources or ingredients consumed in an intervention and the valuation of each ingredient. The specification of ingredients is often facilitated by dividing ingredients into categories including 1) personnel, 2) facilities, 3) equipment and materials, 4) other program inputs, and 5) beneficiary or client inputs.
For Bank-funded projects, public sector costs are identified in project documents and the ingredient method is most useful in identifying other costs such as operation and maintenance costs and non-public sector societal costs.
Identifying costs in the project
Here are some of the costs the project must identify:
First: public sector costs included for the project investments
Project documents lay out investments and non-recurring costs explicitly as part of project preparation and there should be consistency between what is noted in the project and what is identified as project costs, including costs that are part of the loan as well as counterpart costs.
Second: additional operational and maintenance costs or other recurring costs to the public sector
If the project establishes new government entities or invests in new infrastructure that requires funds for operation and maintenance or other recurring costs beyond the project timeframe these should be included. These are incremental costs beyond what would have been paid for in operation and maintenance without the project.
Third: costs to society that should be incorporated - non-governmental organizations, firms, households or individuals
Any additional costs to non-public entities that result from the project should be included. In general, all inputs (costs) that contribute to the generation of benefits should be included. Otherwise, benefits would have to be adjusted accordingly. These costs depend largely on the project and linked to the response required by project beneficiaries to obtain the benefit of the project.
Fourth: total flows of costs in the project
All types of costs should be noted and aggregated to get the total cost flows and the present value of those cost flows. It should be clear that all of these costs are incremental and the manner in which they are measured should be transparent. In the case of shared or common costs (with other projects or among components), these should be distributed and assigned according to a clear allocation rule, which needs to be explicitly stated in the assumptions section.
Once costs and benefits have been identified and valued, alternatives need to be assessed based on an appropriate decision criterion. There are three main criteria:
Net Present Value
Net Present Value (NPV) compares all costs with benefits, with all resource flows discounted by the appropriate discount rate. This comparison is presented in net present value terms typically the current year. If positive, the project should be accepted, if negative rejected on economic merit. The alternative/project with the highest NPV should be selected.
Benefit-Cost (Or Cost-Benefit) Ratio
The Benefit-Cost Ratio (BCR) is the ratio of the sum of benefits (in present value) to the sum of costs (in present value). Typically if this ratio is more than one, using this criterion the project would be approved.
Economic Rate of Return
The Economic Rate of Return (ERR) is defined as the discount rate that causes the NPV to be zero. Typically, if the ERR is larger than the recommended 12% real discount rate the project would be approved.
In practice, it is helpful to compute and present all three criteria. The computation of the ERR is required and should be reported. Neither the ERR nor the BCR provides any indication of the project scale so presenting the NPV is desirable. Based on the presentation of results using the chosen investment criteria, an initial conclusion should be drawn on the viability of the project.
Sensitivity analysis allows the testing, under different scenarios, of the strength and robustness of the assumptions, the data and the logic of the intervention that underlie the economic analysis and to acknowledge the underlying uncertainty of the expected results. Any forward looking estimation presumes behavioral assumptions, assumptions on the vertical logic, and on the causality links that allow the attribution of a particular benefit or cost to the goods and/or services delivered or used by the project.
In addition it is necessary to undertake sensitivity analysis to compensate for the natural tendency for project appraisers to be overly optimistic. Many project parameters are affected by optimism, overstating benefits and understating timings and costs, both capital and operational.
- In many instances both costs and benefits are driven by a few critical factors. Identifying these factors – that can be called “drivers” - is a central element in performing a good sensitivity analysis. The assumptions behind the future performance of these drivers need to be critically assessed and the sensitivity analysis can illuminate at which points in these assumptions the project is no longer worth pursuing. These values are typically called “switching values¨.
- In a sensitivity analysis, it is important to focus on “what if” situations, particularly in adverse or more stringent and acid scenarios than those assumed under a Base Case scenario. This focus will allow the analysis to illuminate those values in critical variables and assumptions that might “switch” the recommendation from Go to No Go (or vice-versa) or to reformulate project design and / or components in order to make the project viable. As there are many sources of uncertainty and numerous risks, it can be very useful to identify the key sources of uncertainty and base the analysis on those. Sometimes having too much information does not provide guidance as to the viability of the project under different situations.
- The sensitivity analysis, which is really a risk assessment, should be coherent with the Risk Matrix and other risks discussed in the project proposal.
- Many projects face significant start and implementation delays. If warranted, a simulation should be undertaken delaying start and implementation dates.
- In a CBA derived from impact evaluations – sensitivity can be done by ranking alternatives by their relative net benefit indicators (NPV, ERR or C/B) using the point estimate of their impact, and then re-compute this ratio using the lower and upper bounds of the impact estimate.
A Base Case reflects the most plausible estimates for unknown quantities and prices and should reflect the average expectation of their future behavior, particularly of cost and benefit drivers. This average expected behavior should be empirically based. The more useful sensitivity analysis are those that present few explicit and reasoned changes on the assumptions of the most critical values of the drivers of the dominant benefits and costs, many times in a “best case, worse case” situation.
- Sensitivity analysis can be “one way” or “multi-way”.
- One-way sensitivity analysis allows for the variation of one critical variable at a time, holding everything else constant. Some authors call this “partial sensitivity analysis¨. This kind of analysis is most appropriately applied to situations in which the analyst believes there is one critical driver.
- Multi-way allows for simultaneous variations of more than one variable at a time. In this type of analysis, variable combinations can be combined into best-case and worst-case scenarios. Some projects lend themselves to simulations, where combinations of different scenarios and their probability of occurrence can be modeled and estimations of critical values can be synthesized in terms of Expectations or Expected Values. This is typically undertaken with decision trees and Monte Carlo Simulations. In these cases, means and variances can indicate project risk profiles.
- In a single-factor sensitivity analysis, it is generally assumed that all other factors stay constant. In many projects, this might not be true and the variables might not be independent. In those cases, a discussion on correlation issues is warranted.
Sensitivity analyses can also help, if the feasibility of a project is very sensitive to a particular assumption on the value of a variable that is uncertain, in identifying mitigating actions that should be considered. If there is exceptional uncertainty, the project might have to be redesigned if implemented on a pilot basis.
Sensitivity analysis is most useful if the proposed scenarios and adjustments are empirically based and/or reflect the unique characteristics of the project in hand—how good and reliable is the data. As with the initial assumptions used in an analysis, ideally, the scenarios are empirically based.
Eliciting WTP through approaches such as contingent valuation is commonly used in benefit estimation and is based on the use of surveys to determine WTP. Although there are many approaches to contingent valuation, and a significant body of literature that explores potential biases, the critical elements are usually associated with survey instrument design and application, valuation methods (referendum and non-referendum models), and payment vehicles.
If the estimation is direct, the WTP can be derived from one observation, combined with previous research that indicates functional form and elasticity. WTP can also be derived from two or more observations, where extrapolation, model specification, and data consistency are the critical elements in the validity of the benefit estimation.
WTP can also be estimated indirectly by various methods such as analogous pricing, trade-off method, hedonic pricing, travel costs, avoided costs, and others.
- When using analogous pricing, the market price for an analogous good or service can be used as a shadow price for a publicly provided good or service.
- Under the trade-off method, the economic analysis uses opportunity cost as a measure of value.
- Hedonic pricing can be estimated when: (i) the relationship between asset price and asset attribute can be established and (ii) an estimation of WTP for each attribute is also possible.
- Under the travel cost method – mostly used in recreational projects – total costs replace entrance price or fee as an explanatory variable for the demand equation.
- The estimation of avoided costs – also called defensive expenditures – is an approximation to benefits assuming that the intervention reduces a current expenditure.
Impact evaluation of previous projects is sometimes helpful when selecting alternatives, establishing causality and attribution, and calibrating expectations of impacts expected from the project. If used, monetary values are placed, under reasonable assumptions, on the estimated – or expected – the impact of the project. This impact is derived using one of the methods – experimental, non-experimental, before and after, and simple difference. These impacts are then expressed in monetary terms.
Most impact evaluations focus on very few (usually one or two) impact indicators and, as a result, the chosen outcome measure may not reflect the full set of impacts nor the corresponding benefits. Along with the impact indicators, other potential project benefits should be incorporated as well as benefits that accrue to non-participants.
In the cases where impact evaluations report significant spillover effects on non-participants, an assessment has to be made on whether spillovers would take place when programs are scaled up, and whether the benefits attributed to these spillovers should be included or not. The use of impact evaluations is more useful in sectors, such as health and education, where there is ample evidence derived from impact evaluations and relevant literature that connects the outcome measures with future benefits.
The estimation of benefits is what separates CBA and CEA and the analysis presented here requires explicitly noting all of the benefits to be included in the analysis as well as the monetary values of those benefits. All relevant project benefits should be noted. Since a societal perspective is taken in CBA for Bank-funded projects, the benefits should include all societal benefits.
The choice of method will depend on many factors. The critical factors are four:
First
Whether a market exists or does not for the goods and services to be provided by the project. If there is a market, price can be elicited directly from published sources or from direct observation. If not, price information will have to be derived from either revealed or stated preferences.
Second
The nature of the Theory of Change. The logic of the intervention and the identified pathways through which the proposed intervention brings the expected impact are central in identifying the mechanisms by which those impacts can be monetized. A credible transition from outputs to outcomes and impacts will hinge on the quality of the assumptions embedded in the Theory of Change and this sequence and logic needs to be consistent with the step required to monetize impacts.
Third
The quality, availability and timeliness (how recent it is?) of the information/data required for benefit estimation.
Fourth
Sector-specific considerations and standard professional practice. Irrespective of the method, benefit estimation should be clearly linked to the logic chain: input→ output→ outcome→ impact (benefits) chain.
For more information, visit the FAQ at the top of this page.
Cost Benefit Analysis in the Context of the Energy Infrastructure Package (Nils-Henrik von der Fehr - THINK-European Union, 2012)
Cost-Effectiveness Analysis of education and Health Interventions in Developing Countries(McEwan, Patrick, J. -IADB, 2011)
Can Cost-Benefit Analysis Guide Education policy in Developing Countries? (Jimenez, Emmanuel; Patrinos, Harry Anthony - World Bank, 2008)
A Review of the Valuation Of Environmental Costs and Benefits in World Bank (Patricia Silva, Stefano Pagiola- World Bank, 2005)
Cost-benefit Analysis of Natural Disaster Risk Management in Developing Countries(International Finance Corporation, Manual 2005)
There are different approaches in identifying costs including the ingredient method which relies on the identification of all resources or ingredients consumed in an intervention and the valuation of each ingredient. The specification of ingredients is often facilitated by dividing ingredients into categories including 1) personnel, 2) facilities, 3) equipment and materials, 4) other program inputs, and 5) beneficiary or client inputs.
For Bank-funded projects, public sector costs are identified in project documents and the ingredient method is most useful in identifying other costs such as operation and maintenance costs and non-public sector societal costs.
Identifying costs in the project
Here are some of the costs the project must identify:
First: public sector costs included for the project investments
Project documents lay out investments and non-recurring costs explicitly as part of project preparation and there should be consistency between what is noted in the project and what is identified as project costs, including costs that are part of the loan as well as counterpart costs.
Second: additional operational and maintenance costs or other recurring costs to the public sector
If the project establishes new government entities or invests in new infrastructure that requires funds for operation and maintenance or other recurring costs beyond the project timeframe these should be included. These are incremental costs beyond what would have been paid for in operation and maintenance without the project.
Third: costs to society that should be incorporated - non-governmental organizations, firms, households or individuals
Any additional costs to non-public entities that result from the project should be included. In general, all inputs (costs) that contribute to the generation of benefits should be included. Otherwise, benefits would have to be adjusted accordingly. These costs depend largely on the project and linked to the response required by project beneficiaries to obtain the benefit of the project.
Fourth: total flows of costs in the project
All types of costs should be noted and aggregated to get the total cost flows and the present value of those cost flows. It should be clear that all of these costs are incremental and the manner in which they are measured should be transparent. In the case of shared or common costs (with other projects or among components), these should be distributed and assigned according to a clear allocation rule, which needs to be explicitly stated in the assumptions section.
Once costs and benefits have been identified and valued, alternatives need to be assessed based on an appropriate decision criterion. There are three main criteria:
Net Present Value
Net Present Value (NPV) compares all costs with benefits, with all resource flows discounted by the appropriate discount rate. This comparison is presented in net present value terms typically the current year. If positive, the project should be accepted, if negative rejected on economic merit. The alternative/project with the highest NPV should be selected.
Benefit-Cost (Or Cost-Benefit) Ratio
The Benefit-Cost Ratio (BCR) is the ratio of the sum of benefits (in present value) to the sum of costs (in present value). Typically if this ratio is more than one, using this criterion the project would be approved.
Economic Rate of Return
The Economic Rate of Return (ERR) is defined as the discount rate that causes the NPV to be zero. Typically, if the ERR is larger than the recommended 12% real discount rate the project would be approved.
In practice, it is helpful to compute and present all three criteria. The computation of the ERR is required and should be reported. Neither the ERR nor the BCR provides any indication of the project scale so presenting the NPV is desirable. Based on the presentation of results using the chosen investment criteria, an initial conclusion should be drawn on the viability of the project.
Sensitivity analysis allows the testing, under different scenarios, of the strength and robustness of the assumptions, the data and the logic of the intervention that underlie the economic analysis and to acknowledge the underlying uncertainty of the expected results. Any forward looking estimation presumes behavioral assumptions, assumptions on the vertical logic, and on the causality links that allow the attribution of a particular benefit or cost to the goods and/or services delivered or used by the project.
In addition it is necessary to undertake sensitivity analysis to compensate for the natural tendency for project appraisers to be overly optimistic. Many project parameters are affected by optimism, overstating benefits and understating timings and costs, both capital and operational.
- In many instances both costs and benefits are driven by a few critical factors. Identifying these factors – that can be called “drivers” - is a central element in performing a good sensitivity analysis. The assumptions behind the future performance of these drivers need to be critically assessed and the sensitivity analysis can illuminate at which points in these assumptions the project is no longer worth pursuing. These values are typically called “switching values¨.
- In a sensitivity analysis, it is important to focus on “what if” situations, particularly in adverse or more stringent and acid scenarios than those assumed under a Base Case scenario. This focus will allow the analysis to illuminate those values in critical variables and assumptions that might “switch” the recommendation from Go to No Go (or vice-versa) or to reformulate project design and / or components in order to make the project viable. As there are many sources of uncertainty and numerous risks, it can be very useful to identify the key sources of uncertainty and base the analysis on those. Sometimes having too much information does not provide guidance as to the viability of the project under different situations.
- The sensitivity analysis, which is really a risk assessment, should be coherent with the Risk Matrix and other risks discussed in the project proposal.
- Many projects face significant start and implementation delays. If warranted, a simulation should be undertaken delaying start and implementation dates.
- In a CBA derived from impact evaluations – sensitivity can be done by ranking alternatives by their relative net benefit indicators (NPV, ERR or C/B) using the point estimate of their impact, and then re-compute this ratio using the lower and upper bounds of the impact estimate.
A Base Case reflects the most plausible estimates for unknown quantities and prices and should reflect the average expectation of their future behavior, particularly of cost and benefit drivers. This average expected behavior should be empirically based. The more useful sensitivity analysis are those that present few explicit and reasoned changes on the assumptions of the most critical values of the drivers of the dominant benefits and costs, many times in a “best case, worse case” situation.
- Sensitivity analysis can be “one way” or “multi-way”.
- One-way sensitivity analysis allows for the variation of one critical variable at a time, holding everything else constant. Some authors call this “partial sensitivity analysis¨. This kind of analysis is most appropriately applied to situations in which the analyst believes there is one critical driver.
- Multi-way allows for simultaneous variations of more than one variable at a time. In this type of analysis, variable combinations can be combined into best-case and worst-case scenarios. Some projects lend themselves to simulations, where combinations of different scenarios and their probability of occurrence can be modeled and estimations of critical values can be synthesized in terms of Expectations or Expected Values. This is typically undertaken with decision trees and Monte Carlo Simulations. In these cases, means and variances can indicate project risk profiles.
- In a single-factor sensitivity analysis, it is generally assumed that all other factors stay constant. In many projects, this might not be true and the variables might not be independent. In those cases, a discussion on correlation issues is warranted.
Sensitivity analyses can also help, if the feasibility of a project is very sensitive to a particular assumption on the value of a variable that is uncertain, in identifying mitigating actions that should be considered. If there is exceptional uncertainty, the project might have to be redesigned if implemented on a pilot basis.
Sensitivity analysis is most useful if the proposed scenarios and adjustments are empirically based and/or reflect the unique characteristics of the project in hand—how good and reliable is the data. As with the initial assumptions used in an analysis, ideally, the scenarios are empirically based.
Eliciting WTP through approaches such as contingent valuation is commonly used in benefit estimation and is based on the use of surveys to determine WTP. Although there are many approaches to contingent valuation, and a significant body of literature that explores potential biases, the critical elements are usually associated with survey instrument design and application, valuation methods (referendum and non-referendum models), and payment vehicles.
If the estimation is direct, the WTP can be derived from one observation, combined with previous research that indicates functional form and elasticity. WTP can also be derived from two or more observations, where extrapolation, model specification, and data consistency are the critical elements in the validity of the benefit estimation.
WTP can also be estimated indirectly by various methods such as analogous pricing, trade-off method, hedonic pricing, travel costs, avoided costs, and others.
- When using analogous pricing, the market price for an analogous good or service can be used as a shadow price for a publicly provided good or service.
- Under the trade-off method, the economic analysis uses opportunity cost as a measure of value.
- Hedonic pricing can be estimated when: (i) the relationship between asset price and asset attribute can be established and (ii) an estimation of WTP for each attribute is also possible.
- Under the travel cost method – mostly used in recreational projects – total costs replace entrance price or fee as an explanatory variable for the demand equation.
- The estimation of avoided costs – also called defensive expenditures – is an approximation to benefits assuming that the intervention reduces a current expenditure.
Impact evaluation of previous projects is sometimes helpful when selecting alternatives, establishing causality and attribution, and calibrating expectations of impacts expected from the project. If used, monetary values are placed, under reasonable assumptions, on the estimated – or expected – the impact of the project. This impact is derived using one of the methods – experimental, non-experimental, before and after, and simple difference. These impacts are then expressed in monetary terms.
Most impact evaluations focus on very few (usually one or two) impact indicators and, as a result, the chosen outcome measure may not reflect the full set of impacts nor the corresponding benefits. Along with the impact indicators, other potential project benefits should be incorporated as well as benefits that accrue to non-participants.
In the cases where impact evaluations report significant spillover effects on non-participants, an assessment has to be made on whether spillovers would take place when programs are scaled up, and whether the benefits attributed to these spillovers should be included or not. The use of impact evaluations is more useful in sectors, such as health and education, where there is ample evidence derived from impact evaluations and relevant literature that connects the outcome measures with future benefits.