Effectiveness For Improving Lives

effectiveness for improving lives

ECONOMIC ANALYSIS OVERVIEW

WHAT IS THE OBJECTIVE OF THE PROJECT?

The scope of the objective will determine the choice of the analysis methods and tools. The project’s objective should be linked with the expected impact. As objectives broaden in scope, attribution becomes harder to determine, mainly because multiple causality chains can be present and multiple effects captured.

WHAT IS THE COUNTERFACTUAL?

The most fundamental question in CBA is the establishment of the counterfactual as it is the basis for the “without” project situation. The starting point for the definition of the counterfactual is what can be called “the improved base situation without project”. This situation reflects small improvements to the project at very little cost and can serve as the benchmark against which to compare the proposed project. Also, this counterfactual can be derived from a rigorous impact evaluation that can be validly applied to the project, possibly from a pilot or another similar project that are externally valid to the one under analysis. In most cases, explicit assumptions will need to be made to compensate for the limited information on the counterfactual often relying on a “business as usual” or “improved business as usual” scenario.

WHAT ARE THE COMPONENTS? HOW CRITICAL ARE THEY? HOW SEPARABLE?

The realization of expected project benefits is contingent on the effective application of a combination of inputs, grouped in components. Each component should be justified and analyzed on its own, based on an additional or marginal analysis (with/without component). Nevertheless, some components are more critical than others, and if benefits can be more clearly identified, quantified and monetized for one or several of these components, then the economic analysis can be applied to those critical components.

In some cases, benefits are the result of the combination of components and attribution to a specific component is not possible. When undertaking a CBA that is based on only one of the components of the projects, special attention should be placed on benefits attribution of that component.

WHY IS THIS PROJECT THE BEST ALTERNATIVE TO REACH THE OBJECTIVE?

In the selection of the alternative (s), it is important to consider any plausible mutually exclusive alternatives that involve differing technological/institutional choices, financial arrangements, locations and beneficiaries. These alternatives can generate different net benefit streams and CBA allows the selection of the best available option.

DID THE ANALYSIS SERVE IN CHANGING THE PROPOSED PROJECT?

Economic analysis is a tool that is designed to help select projects. It is most useful if used early in the project cycle to identify bad alternatives and bad components. If used late in the cycle, its usefulness is restricted to helping decide whether to proceed or not with a particular design. If the analysis influenced the design of the project, it should be noted. If not, particular attention will be given to critical assumptions when no additional alternatives are presented as analyzing a particular project is significantly different from justifying it.

DIFFERENT MANNERS OF MEASURING INCREMENTAL OUTCOMES OR IMPACTS

In CEA, while the cost is monetary unit, the incremental effects are expressed in non-monetary terms. The result is typically a ratio, cost per effect.


In CBA the incremental benefits or net social benefits, are associated with the gain in social surplus generated by the intervention or project. These incremental net benefits are expressed in monetary terms. The result is typically expressed as a monetary value such as Net Present Value (NPV) or as a rate at which that NPV switches from positive (go) to negative (no go).

HAVING A MONETARY UNIT AS A UNIT OF ACCOUNT OR NUMERAIRE FOR CBA

CEA compares (mutually exclusive) alternatives in terms of their cost per effect, and this effect has to be common to all alternatives. One cannot compare two interventions with different effects, or the same effect measured in distinct ways. For example, an intervention that affects school attendance cannot be compared with an intervention that affects class size unless both their impacts can be measured in increases in test scores.

In CBA, as all benefits and costs are expressed in common monetary terms, alternatives with differing effects can be compared and ranked by their net benefits: the Net Present Value of a road can be compared with the Net Present Value of a school.

OTHER THREE DIFFERENCES BETWEEN CBA AND CEA
  • Number of effects: Projects or interventions can have more than one effect. CEA is restricted to capturing only one effect (there will be a different CE ratio for each effect), while CBA can encapsulate more than one effect.
  • Number of assumptions: CEA will typically derive its ratios from specific interventions, and thus the number of assumptions that need to be made are limited by the intervention under consideration. On the other hand, CBA has to go a step further and put a monetary value on all effect requiring additional assumptions (e.g. prices). The number of assumptions in CBA tends to be greater.
  • Source of information: In CEA analysis, alternatives are typically derived from impact evaluations that are based on rigorous comparisons of treatment and control groups while in CBA that is not typically the case.
DOUBLE COUNTING BENEFITS

Double counting of benefits can happen in one of two ways.

First, when the value of an intermediate good or service is measured twice. For example, if the benefits of an irrigation project are based on an estimated demand curve, also counting the additional income that accrues to farmers from irrigation is counting the benefits twice.

Second, when the benefit is counted once as a stock and then again as a flow. For example, adding a higher property value that results from reduced travel times to the location along with the monetized benefit of reduced travel times. Higher property values would include, to a large extent, the value of location with regard to travel time; and if you count both you overestimate the project’s benefits.

INCORPORATING TRANSFERS AS BENEFITS (OR COSTS)

Some payments that appear in the financial cost (benefit) streams do not represent direct claims on the country resources, but only reflect a resource control transfer from one sector of the society to another. Loan and interest payments, taxes, subsidies and depreciation allowances all fall into this category. In general, any cost or benefit flow that does not reflect a real resource use should be considered a transfer and netted out when benefits and costs are presented from a societal perspective.

FAILING TO INCORPORATE EXTERNALITIES

In many projects, externalities, particularly environmental externalities, can be a significant source of benefits or costs and should not be ignored. Typically an externality is a by-product of production or consumption that has no market.

DEALING WITH SECOND AND THIRD ROUND EFFECTS

In some cases, a project might affect markets that are one or two steps removed. Caution should be exercised when valuing these benefits. Second and third level effects should only be considered if these markets present distortions.

EMPLOYMENT AS BENEFITS

Some projects have an explicit – or implicit – goal of providing employment. In all cases employment is a cost, not a benefit as it entails the use of a scarce resource. In those cases where employment is pursued, an appropriate shadow wage rate should be used to reflect the real opportunity cost of labor.

LACK OF EVIDENCE FOR PROJECTION RATES

In many projects, future benefits areforecasted using ad-hoc growth rates. The assumptions and sources for the validity of these rates should be made explicit.

COSTS BEYOND THE PROJECT BUDGET

Project costs should be total project costs to society, not only those costs that are borne by the Bank or by counterpart funds.

MIXING NOMINAL AND REAL FLOWS

All analysis should be made in a consistent basis. If the discount rate is real, flows should be presented in real terms. If flows are in nominal terms, nominal discount rates should be used. Nevertheless, it is not recommended to use nominal values.

KEY ASPECTS OF THE METHODOLOGY

Any estimation of costs, and particularly benefits, is based not only on the assumptions on the future evolution of market (and shadow) prices and quantities, but also on the methods used in estimating costs, benefits, and impacts that cannot be derived directly from market valuations. The estimation of benefits, which express in monetary terms the project’s outcomes, can be done in two ways depending on if there is or is not a market for the good or service.

  • If there is market, putting a value to the goods and services is straightforward.
  • If there is not a market for the goods and services, there is a need to proceed with a non-market valuation. Approaches to do this include revealed preferences (e.g. travel cost, hedonic pricing) and stated preferences (e.g. contingent valuation).

In many specific markets, it might be important to correct for significant price distortions that can range from foreign exchange under or over valuation, traded goods and non-traded (for example labor market) disequilibria, or market prices that do not reflect true economic opportunity costs as they include taxes and subsidies.

DISCOUNTING FUTURE VALUES

Projects require that future benefit and costs flows be discounted to account for the opportunity cost of capital. It is recommended that a 12% real discount rate be used in all of Bank projects. In certain cases, where benefits accrue in the very long term, sensitivity analysis can be performed simulating lower real discount rates,provide that the rates are consistent with best practices in the specific technical literature.

ANALYSIS PERIOD

Typically the investment period of a Bank project will range between 1 and 5 years. Benefits accrue over a longer period. The definition of the evaluation period depends on whether benefits are temporary or permanent. In the case of temporary benefits (typical in hard infrastructure investments), the evaluation period should be consistent with the expected life of the investment (replacement). In the case of permanent benefits (more prevalent in health and education) the evaluation period will depend on the expected duration of the estimated impacts (or benefits, if monetized).

The length of the evaluation period should be noted and justified. In practice, the relevant horizon is obviously affected by the discount rate used. With a 12% discount rate, a horizon of more than 20 years is probably not going to impact your NPV calculation.