EVO - Evaluation- A Management Tool for Improving Project Performance (a logical framework) -3/97-

The logical framework is a tool for the conceptualization, design, execution and evaluation of projects. Its purpose is to give structure to the process of planning and to communicate essential information about the project. It can be used at all stages of project preparation: programming, identification, orientation, analysis, presentation to review committees, execution, and ex-post evaluation. It should be developed with early participation of the country team, and then detailed participation of the borrower, the borrower's consultants, the project team, the Country Office and the executor. It is modified and improved iteratively during both preparation and execution.

The logical framework was originally developed in response to three commonly-found problems:  

1. project planning lacking precision, with multiple objectives not clearly related to project activities;
2. projects not being executed successfully, and the extent of the manager's responsibility was not clear; and,
3. there was no clear picture of what a particular project would look like if it were successful, and evaluators had no objective basis to compare what was planned with what actually happened.

The logical framework method addresses these problems by stressing clarity of objectives, determination of responsibilities and measurability. It provides a number of advantages over less structured approaches:

1. a uniform terminology that facilitates communication and helps reduce ambiguity ;
2. a format to reach agreements about project objectives, targets and risks that are shared by the Bank, borrower and executor;
3. a common analytic outline that can be used by the borrower, consultants, and the project team to develop both the project and the project report;
4. focus to technical work on critical aspects and can shorten Profiles and Project documents considerably;
5. the information to organize and prepare the project execution plan;
6. the necessary information for execution, monitoring, and evaluation all in one place.
7. a structure to present the most important information about the project in a single table.

The logical framework is presented as a four by four matrix (see Figure 1). The four columns provide the following information: 

1. A Narrative Summary of the Objectives and Activities
2. Indicators (specific targets to be reached)
3. Means of Verification
4. Assumptions (external factors that imply risks)

The rows of the matrix present information on objectives, indicators, means of verification, and assumptions at four different times in the life of the project: 

1. Goal to which the project contributes significantly after the project has been in operation.
2. Purpose achieved when the project has been completed.
3. Outputs/components achieved during execution.
4. Activities required during production of the Outputs.

Every Bank project is the response to a problem that has been detected. The Goal is a statement of the solution to the problem that has been diagnosed. If, for example, the principle problem in the health sector is a high rate of maternal and infant mortality in the low income population, the objective would be to reduce the rate of maternal and infant mortality in that population.

Two things should be emphasized about the Goal. First, it does not imply that the project, by itself, will be sufficient to attain the Goal. It is sufficient that the project contribute significantly to attaining the Goal. Second, the statement of the Goal does not imply that it will be realized shortly after the project is in operation. It is a long term objective to which the project will contribute.

Ideally the Goal should be identified by the Country Team, in consultation with the stakeholders or interested parties, during the preparation of the Country Programming Paper and the Programming Mission and made precise during the Identification Mission. Various projects or actions could contribute to the solution of the problems that have been identified. It is unlikely that one project, by itself, will solve the problem or problems. In the context of the Logical Framework approach, the purpose of the Identification Mission is to identify which, of a number of alternative projects, would contribute most (or most-cost effectively) to the solution.

The Purpose is the result expected at the end of the execution period. It is the change that the project will foster. It is a hypothesis about what should occur as a consequence of producing and utilizing the outputs. The title of the project should come directly from the statement of the Purpose. The logical framework requires that each project have only one Purpose. The reason for this is clarity. If there is more than one Purpose there is ambiguity, and trade-offs can arise. The executor may choose to pursue the Purpose that he views as most important, or easiest to achieve, or least expensive. This, however, may not be the Purpose that the Borrowing Country and the Bank thought were the most important.

Given that it is a hypothesis, it is important to recognize that the achievement of the Purpose is beyond the control of the project management or executor. The project management is responsible for producing Outputs (the physical works, the technical cooperations, and the training). However, other groups must use these Outputs to achieve the project's Purpose. These groups are beyond the project management's control.

For example, in an irrigation project, the project management is responsible for building irrigation works and making sure that water flows through them. He may be responsible for instructing farmers on how to use the water and how to grow crops with the water. He cannot be responsible for the project's Purpose: increased agricultural production. This is beyond his control. The farmers may be unwilling to change their practices; there may be a drought that reduces the water available to the works; a disease or pest may attack the area. The project management cannot be responsible for these.

Many operations that the Bank finances have more than one Purpose. When this is the case, the approach requires more than one logical framework: a master framework for the Program with its Goal and Purpose, comprised of two or more projects. The Goal of each of the subordinate logical frameworks is identical to the Purpose of the Program (master framework), but each project has its specific Purpose. Each separate Purpose is addressed by the Program with its own logical framework.

Outputs are the specific works, studies, services, and training that the project management is required to produce with the budget they are given. Each of the Outputs must be necessary to achieve the Purpose, and it should be reasonable to think that if the Outputs are produced, the Purpose will be achieved. The project management is responsible for producing the Outputs of the project. The Outputs are the subject of the loan contract. They must be clearly stated. In the logical framework, the Outputs are stated as results, that is, as work completed, studies finished (and possibly conclusions implemented), training completed.

The activities are the tasks that the executor must undertake to produce each Output. A detailed list of activities is important because it is the point of departure for the plan of execution. Each activity is listed in a Gantt chart and the time and resources required for its execution is estimated. Hence, execution is directly linked to project design.

The logical framework is constructed so that managers can examine the causal links from bottom to top. If the project is well designed, the following are true: 

• the Activities listed for each Output are necessary to produce the Output;
• each Output is necessary to achieve the project's Purpose;
• no Output necessary to achieve the project's Purpose is missing;
• if the Purpose of the project is achieved, it will contribute to the attainment of the Goal;
• the Goal, Purpose, Outputs and Activities are clearly stated;
• the Goal is a response to the most important problem in the sector.

Indicators make the intended results specific in three dimensions: quantity, quality, and time. The Purpose of a project might be to bring the sanitary conditions of the coastal waters into compliance with health standards and the indicator might be to reduce the average count for total coliforms to below 1000 per 100 ml on a specific beach by 2001. Such an indicator is unambiguous. It focuses the project management on a quantitative, measurable target, less than 1,000 per 100 ml of total coliforms, rather than possible alternatives (faecal coliform or enterococci). It specifies quality (in this case the beach where the result is to be achieved), and it tells when the results are expected.

Although there may be many potential indicators of expected results, the logical framework should specify the minimum number necessary to conclude that the Purpose has been achieved. The indicators should measure change that can be attributed to the project, and should be obtainable at reasonable cost, preferably from existing data sources. The best indicators contribute to ensuring good project management and allow project managements to decide if additional Outputs or corrections will be necessary to achieve the project Purpose.

In some projects, such as those in the social sectors, it may be difficult to find measurable indicators. Sometimes, it may be necessary to use proxies. However, the availability of measurable indicators should obviously not determine project design. As E. J. Mishan once pointed out, "It is better to have a crude measure of the right concept, than a perfect measure of the wrong one."

The indicators for Outputs are succinct descriptions of the physical works, studies, and training provided by the project. The description should specify quantity, quality and time. For example, an education project might specify 10 technical schools, located in specified cities, each with a capacity of 1,000 students per year, and equipped as specified by law (or listed in an annex to the project report).

The project budget appears as the indicator of Inputs/Activities. The budget is presented by the cluster of activities representing an Output.

In reviewing the indicator's column, managers should check to see that:  

1. indicators of Purpose are not a summary of the Outputs, but a measure of the result of having the Outputs in operation;
2. indicators of Purpose measure what is important;
3. all indicators are specified in terms of quantity, quality and time;
4. indicators at each level are different from the indicators at the other levels;
5. the budget is sufficient to carry out the Activities identified.

The logical framework tells where the executor or evaluator can obtain information on the indicators. This forces the project planners to identify existing sources of information or to make provision to collect information, perhaps as a project input/activity. Not all information has to be statistical. Production of Outputs can be verified by visual inspection by the specialist. Execution of the budget can be verified from receipts submitted for reimbursement or as justification for replenishment of the rotating fund.

Every project involves risks: environmental, financial, institutional, social, political, climatological or other factors that could cause the project to fail. The logical framework requires that the project design team identify the risks at each stage: Activity, Output, Purpose, and Goal. The risk is stated as an assumption that must hold in order to proceed to the next level in the hierarchy of objectives. The reasoning is the following: if we carry out the activities listed and certain assumptions hold, then we will produce the Outputs listed. If we produce the Outputs and other assumptions hold, then we will achieve the project's Purpose. If we achieve the project's Purpose, and still further assumptions hold, then we will contribute to the achievement of the Goal. The assumptions represent a judgement on the probability of project success that is shared by the project design team, the borrower, the Country Office, Bank management, and the executor, who should be involved in the process of project design.

The project risks have one important characteristic: the risks are defined as being outside the direct control of the project management. The project design team asks itself what could go wrong at each level. At the Activity level, for example, counterpart funds might not arrive in a timely manner, or there might be a change in government priorities, or a strike, or a major devaluation, etc. The objective is not to list every conceivable contingency, but to identify reasonably likely possibilities.

The assumptions column plays an important role in both planning and in execution. At the planning stage it helps identify risks that can be avoided by incorporating additional components in the project itself. For example, one of the assumptions of a rural health program might be that qualified personnel will be willing to move to and live in rural areas. Since this assumption is critical to achieving the project's Purpose, the willingness of personnel should not be left to chance. The project design team should build into project design, Activities and Outputs to assure that qualified personnel will be willing to locate in rural areas. Such Outputs might be a system of incentive payments, provision of a free residence, or something else. If the assumption is critical, and there is no Output that can control it, the project design team and management may well decide that the particular project is too risky and should be abandoned.

The assumptions are important during execution as well. They indicate the factors that the project management should anticipate, try to influence, and address with contingency plans. For example, if the success of an irrigation project assumes that the Ministry of Agriculture will fulfill its commitment to remove price ceilings or export restrictions by a certain date, the project management should constantly monitor progress and point out to the Ministry and to the Bank that delays may frustrate the achievement of the project's Purpose. One of the advantages of having the assumptions stated is that the project management has an incentive to communicate problems rather than leaving them for someone else to discover.

The execution plan is not part of the logical framework, but an annex to it. It takes the list of Activities necessary in the logical framework, disaggregates these to the task level, and presents a chart of the start and completion time of each task and Activity (a Gantt chart). It also assigns responsibility for carrying out each activity to an organization or individual. The scheduling of Activities allows the design team, management and the project management to assess whether the time given to complete the project is realistic. It also allows the project management to distribute and coordinate tasks when many institutions are participating.

The logical framework is a tool, not a static device for presentation of final results. It is modified and completed in the process of project preparation. It has the potential to focus and make more efficient the project preparation process at the Bank. It can contribute at every stage in the the project cycle. If the approach is used, the logical framework is developed sequentially.

In the process of preparation on the Programming Document, the Country Team, Country Office and Planning Office of the country focus only on the problem to be solved: they concentrate on the statement of the Goal, how it will be measured, and where the information on the indicators of the problem can be found (i.e., the first two cells of the first row of the logical framework shaded in Figure 1).

During identification the Borrower, Country Office, and the technical working group look at alternative projects that could contribute to the solution of the problem, and the resulting Profile I would select the best alternative and define its Purpose and indicators. They would concentrate of the statement of the Goal and Purpose of the project, the indicators, and the sources of information (the shaded area of Figure 2). At this stage, important aspects of the project begin to be considered. The environmental analysis makes a preliminary assessment of the cost of mitigation and of the possible damages resulting from different project alternatives. The economic analysis attempts to determine the least cost alternative. The institutional analysis identifies the most viable ways to execute and operate the project, and the financial analysis examines the capacity of the executor to provide counterpart financing and generate sufficient funds to sustain the project's operation. The working group considers the Bank's replenishment objectives for the environment, women, the low income population, and other aspects such as financial targets and institutional reform. When these are relevant, they appear as indicators of Purpose (see the area marked with asterisks in Figure 2).

During Orientation, the Project Team concentrates on dimensioning the project which implies establishing the relation between what will be achieved (the Purpose indicators) and the number, type, size and cost of the project's Outputs. At the level of Profile II, the Project Team may modify the indicators, but they should not modify the Goal and Purpose. If the Goal and Purpose change, then the project has changed and a new Profile I should be prepared, or the change should be highlighted in Profile II.

Profile II should include the Outputs, their indicators, and the assumptions at the levels of Goal, Purpose, and Outputs. Profile II should present information corresponding to the shaded parts of the logical framework presented in Figure 3. The work effort at the level of Profile II focuses on the quantitative and qualitative relationship between the Output of the project and the results to be achieved and the risks (the part of Figure 3 indicated with ). Much of the essence of the financial and institutional analysis is summarized under assumptions. For example, the financial viability may depend on an increase in tariffs (a contractual condition, the compliance with which is an assumption) or the institutional viability may depend on close coordination between several ministries, which is also an assumption.

The logical framework constitutes the outline for the borrower's proposal and the Bank's project report. It follows the same logic: identification of the principal problem, selection of the best project alternative to contribute to the solution (the Goal), statement of project Purpose (expected results at the end of project execution), a list of the Outputs necessary to achieve the Purpose, and a list of the Activities necessary to produce the Outputs, the cost of the Activities (the budget), and Assumptions (risks). The logical framework highlights any aspects that should be developed in more detail in complementary evaluations or that should be supported in the report. The evaluations help determine the best alternative to manage the risks of the project. The evaluations include diagnoses of the institutional capacity to carry out the activities, the environmental impact of the Activities and the Outputs, the financial capacity to carry out and operate the project, and the economic feasibility.

The logical framework contains a summary of almost all pertinent information provided by multi-disciplinary analyses. The technical analysis is summarized in the indicators at the Purpose level, in the indicators at the Output level, in the indicators of the Input level (in the budget), and in the execution plan. The IDB-8 priority areas (poverty, women in development, the environment) are reflected in the indicators at the Goal and Purpose levels, and in certain Outputs generated by corresponding activities. The economic analysis emanates from the comparison of the indicators at the Goal or Purpose levels and the budget. The results of the institutional and financial analysis can appear as indicators at the Goal or Output levels (such as training completed, studies conducted, equipment procured), or as Assumptions (which may be included as contractual conditions).

At the stage of presentation to Bank committees, the logical framework provides an executive summary: a succinct overview of the objectives, expected results, costs and risks.

For project execution, the logical framework and plan of execution are the basic documents. They are short enough that everyone can have a copy. All participants know specifically what they are trying to achieve, how success will be measured, who is supposed to do what, when they are supposed to do it, and what risks must be monitored and anticipated.

As has already been stated, the achievement of the Purpose and Goal are beyond the control of the project management or the executor. The project management only has control of the Activities, budget and Outputs that are delivered (see Figure 6). There are factors outside his control, such as the policy and the physical environment. However, the project management has the responsibility to follow these external factors that have been identified in advance as risk factors in advance, to report to supervising authorities when it appears that assumptions about these external factors are not going to hold, and to suggest actions that will permit the achievement of the project's Purpose.

At the stage of ex-post evaluation, evaluators will know what results were expected and where they can find information to verify whether the results were achieved.

1. The Goal is clearly stated.
2. The Goal indicators are verifiable in terms of quantity, quality and time.
3. The project has one Purpose.
4. The Purpose is clearly stated.
5. The Purpose indicators are not a summary of Outputs, but a measure of the achievement of the Purpose.
6. The Purpose indicators measure what is important.
7. The Purpose indicators have quantity, quality, and time measures.
8. The Purpose indicators measure the results expected at the end of project execution.
9. The project's Outputs are clearly stated.
10. The Outputs are stated as results.
11. All Outputs are necessary for accomplishing the Purpose.
12. The Outputs include every item for which the project management is responsible.
13. The Output indicators are verifiable in terms of quantity, quality and time.
14. The Activities include all the actions necessary to produce each Output.
15. The Activities identify any actions required for gathering information on indicators.
16. The Activities are the tasks for which costs are incurred to produce the Outputs.
17. The relationship between the Activities and budget is realistic.
18. The if/then relationship between the Purpose and Goal is logical and does not skip important steps.
19. The relationship between the Outputs and the Purpose is realistic.
20. The vertical logic among Activities, Outputs, Purpose and Goal is realistic as a whole.
21. The Purpose plus assumptions at that level describe the necessary, although not sufficient conditions for achieving the Goal.
22. The Outputs plus the assumptions at that level, describe the necessary and sufficient conditions for achieving the Purpose.
23. The assumptions at the activity level do not include any tasks that have to be carried out before the other Activities can begin. Conditions precedent are listed separately.
24. The means of verification column identifies where the information for verifying each indicator can be found.
25. The Logical Framework defines the information necessary for project evaluation (e.g. , at project completion, ex-post).

Top