Looking/monitoring for

what the members are

doing

Prepared for: Oromia BIUD-project designs and constructions main

core process , major cities, 18 zones
Prepared by: Temesgen Aklilu, GIZ_UGDP
July,2011
Sabata,
Oromia
 A project is a series of activities (investments) that
aim at solving particular problems within a given time
frame and in a particular location.
 The investments include time, money, human and
material resources.
 Monitoring should take place at and be integrated into
all stages of the project cycle by all
individual/institutions that have an interest.
 The three basic project stages include:
 Project planning (situation analysis, problem
identification, definition of the goal, formulating
strategies, designing a work plan, and budgeting);
 Project implementation (mobilization, utilization and
control of resources and project operation); and
 Project evaluation.
 The questions are:
 Where are we?
 Where do we want to go?
 How do we get there? and
 What happens as we do?
• What is monitoring?: “A continuing function that
uses systematic collection of data on specified
indicators to provide management and the main
stakeholders of an ongoing development
intervention with indications of the extent of
progress and achievement of objectives and
progress in the use of allocated funds”. Or
• Monitoring is the regular observation and recording
of activities taking place in a project or programme.
It is a process of routinely gathering information on
all aspects of the project.
• It embodies the regular tracking of inputs, activities,
outputs, outcomes and impacts of development
activities at the project, program, sector and
national levels.
 What is evaluation?: “the process of determining
the worth or significance of a development
activity, policy or program ….. to determine the
relevance of objectives, the efficacy of design and
implementation, the efficiency or resource use, and
the sustainability of results. An evaluation should
(enable) the incorporation of lessons learned into the
decision-making process”. Or
 Evaluation is a process of judging value on what a
project or programme has achieved particularly in
relation to activities planned and overall objectives. It
involves value judgment and hence it is different
from monitoring (which is observation and reporting
of observations).
 OBJECTIVES OF MONITORING AND EVALUATION

Monitoring and evaluation help staff to...

• Monitoring and evaluation are synergistic
• Monitoring information is a necessary but
not sufficient input to the conduct of
rigorous evaluations
• monitoring information can be collected and
used for ongoing management purposes
• Monitoring typically covers only certain
dimensions of a project’s or program’s
activities.
• evaluation has the potential to provide a
more balanced interpretation of
performance
• evaluation is a more detailed and time-
consuming activity
• Monitoring process needs cost and time
 MONITORING & EVALUATION – COMPARATIVE CHARACTERISTICS

Characteristics Evaluation Monitoring

Subject: usually focused on strategic aspects addresses operational management
issues
Character: incidental, flexible subject & methods continuous, regular, systematic

Primary client: stakeholders and external audience program management

Approach: objectivity, transparency utility

Methodology: rigorous research methodologies, rapid appraisal methods
sophisticated tools

Primary focus: focus on relevancy, outcomes, impact focus on operational efficiency and
and sustainability effectiveness

Objectives: to check outcomes / impact, verify to identify and resolve implementation

developmental hypothesis problems

to document successes and lessons to assess progress towards objectives

learned
 Monitoring and evaluation (M&E) ensure
project accountability, efficiency and
effectiveness.
 Monitoring and evaluation help improve
performance and achieve results.
 More precisely, the overall purpose of
monitoring and evaluation is the
measurement and assessment of
performance in order to more effectively
manage the outcomes and outputs known
as development results. Performance is
defined as progress towards and
achievement of results.
 Checking on how project activities are
progressing
 It is observation-systematic and purposeful
observation.
 It involves giving feedback about the
progress of the project to the donors,
implementers and beneficiaries of the
project.
 Reporting: enables the gathered information
to be used in making decisions for improving
project performance.
 very important in project planning and implementation.
 It is like watching where you are going while riding a
bicycle; you can adjust as you go along and ensure that
you are on the right track.
 Monitoring provides information that will be useful in:
1) Analyzing the situation in the project;
2) Determining whether the inputs in the project are well utilized
3) Identifying problems facing the project and finding solutions;
4) Ensuring all activities are carried out properly by the right
people and in time;
5) Using lessons from one project experience on to another; and
6) Determining whether the way the project was planned is the
most appropriate way of solving the problem at hand.
a) Basic engineering and detailed
drawings
b) Procurement works for bought out
items
c) Manufacturing activities at the
contractor’s and subcontractor’s works
d) Supplies
e) Activities at site
Four types:
1. Input indicators: describe what goes on in
the project (e.g. number of bricks brought
on site and amount of money spent);
2. Output indicators: describe the project
activity (e.g. number of classrooms built);
3. Outcome indicators: describe the product of
the activity (e.g. number of pupils
attending the school); and
4. Impact indicators: measure change in
conditions of the community (e.g. reduced
illiteracy in the community).
Parameters
a. Objectives of the project
b. Sanctioned cost
c. Scheduled date of commissioning
d. Techno-economics
e. Various directives as per sanction letter
Methods of monitoring and evaluation
 choosing between doing it via a meeting or field
visit, or in a more specific way
 participative mapping or comparison study.
Tools
 Monitoring and evaluation tools include data,
maps, photos, reports, questionnaires,
 attendance sheets, etc.
It demonstrates that:
1. Planning describes ways which
implementation and monitoring
should be done;
2. Implementation and monitoring are
guided by the project work plan; and
3. Monitoring provides information for
project planning and
implementation.
 Is important to identify the constraints or
bottlenecks that hinder the project in
achieving its objectives
 Enables the project planners and
implementers to assess the benefits and costs
that accrue to the intended direct and indirect
beneficiaries of the project.
 Is essential for drawing lessons from the
project implementation experience and using
the lessons in the planning of other projects in
that community and elsewhere.
 Should provide a clear picture of the extent to
which the intended objectives of the activities
and the project have been realized.
 Evaluation can and should be done:
(a) Before (pre),
(b) During ( interim), and
(c) After (post) implementation.
a. Before project implementation:
Evaluation is needed in order to:
• Assess the possible consequences of the planned
project
• Make a final decision on what project alternative
should be implemented
• Assist in making decisions on how the project will
be implemented
b. During (interim) project implementation:
 Evaluation should be a continuous process and should take place
in all project implementation activities.
 This enables the project planners and implementers to
progressively review the project strategies according to the
changing circumstances in order to attain the desired activity
and project objectives.
c. After (post) project implementation:
• To retrace the project planning and implementation process, and
results after project implementation
• Identifying constraints or bottlenecks inherent in the
implementation phase;
• Assessing the actual benefits and the number of people who
benefited;
• Providing ideas on the strength of the project, for replication; and
• Providing a clear picture of the extent to which the intended
objectives of the project have been realized.
Types of evaluation include :
a. Formative,
b. Summative,
c. Process,
d. Outcomes and
e. Impact.
1. Formative evaluation focuses on early phases of a project.
2. Summative evaluations determine the effectiveness of a project.
3. Process evaluation assesses a project's implementation.
4. Outcomes evaluation determines the changes resulting from a
project.
5. Impact evaluation describes the long-term effects of a project.

Comprehensive M&E includes all of these forms of evaluation at

the appropriate time. Your logic framework assists in
determining the necessary indicators for each type of
1) Collaborative, Participatory and
Empowerment Evaluation
2) Qualitative Evaluation
3) Quantitative Evaluation
 Bringing stakeholders into the evaluation
process is important in process, outcomes and
impact evaluations.
 Collaborative evaluation requires a substantial
degree of coordination between evaluators and
stakeholders.
 Participatory evaluation fully engages
stakeholders with evaluators acting only as
 Empowerment evaluation gives control of
the evaluation to participants with the
evaluator serving as facilitator.
 Qualitative evaluation renders detail-rich
case studies, personal stories and
experiences. This form of evaluation helps
assess process and human impacts. The
three main tools of qualitative inquiry are
interviews, observations and documents.
 Quantitative evaluation focuses on
statistical and numerical data. Data is
collected through surveys, experiments or
numerical analysis of other sources.
 Evaluation includes…
1. Efficiency Were the local and external resources optimized?

2. Results Were the targeted outputs attained?

3. Timing Were the outputs available at the right time?

4. Quality Was this a good project in the environment where it was used?

5. Pertinence Were the outputs made available to the right people?

6. Acceptability Did the local community use the outputs/services offered?

7. Strategy Did the outputs offered correspond to the needs of the population?

8. Scope How much of the real need was covered? Is that percentage
satisfactory?

9. Impacts/objectives Did the project reduce the vulnerabilities of the population?

10. External aspects How did the environment affect the results of the project?
 Evaluation Capacity Factors
1. Demand for / Interest 2. Supply of quality 3. Organization's ability to
in evaluation evaluation learn from evaluation
findings
1.1 Formal provisions for 2.1 Availability of 3.1 Institutional system
conducting evaluations professional evaluators conducive to using / learning
from evaluations
1.2 Civil society 2.2. Access to proper data 3.2 Skills to apply evaluation
development – Public sources findings in managing,
interest in public sector programming and reporting
performance

1.3 Knowledge & 2.3 Knowledge of evaluation

understanding of evaluation methods and tools
among stakeholders
Some typical ways that an external evaluator performs
monitoring and evaluation:
1. Document Review
The evaluator reviews the project's planning and activity reports. They
then analyze whether
the project has achieved the planned targets and the required
changes.
2. Discussion
To get an accurate picture of the project and its obstacles, the
evaluator will hold discussions
with stakeholders.
3. Interview
Interviews with group members, communities or informal leaders will
give more objective
information that perhaps can't be expressed frankly elsewhere. It is
also a way to gain
important information from the outside.
4. Special monitoring and evaluation meeting
A special meeting for reflection usually involves all relevant
stakeholders
M&E can take place at:
 Community local people level
 Local administration level
 Local district (Woreda/city) level
 Monitoring at zonal/regional/federal
level
 Monitoring at donors level
 Project Input Activity Output Outcome Impact
objective
s (E.G.)
Foundation
and earth
work
Floors, walls,
column,
beams
(structural
works)

Roofing
Plastering
and
painting
Electrical/
sanitation
Other
finishing
 PROJECT’S M&E SYSTEM
EVALUATION
INTERVENTION
SOURCES OF DATA M&E TOOLS FOLLOW
- UP
LOGIC

OVERALL FINAL EVALUATION (FE) FINAL EVAL

. REPORT (FER)
EXTERNAL REPORTS &DATA
OBJECTIVES SURVEYS, STATISTICAL ANALYSIS - PROGRAMMING

MID- TERM EVALUATION (MTE) MID- TERM EVAL

. REPORT (MTE)
BENEFICIARIES’ OPINIONS
BENEFICIARIES SURVEY - REVISING CONTRACTS
PROJECT
PURPOSE

STRUCTURED INTERVIEW / FOCUS QUARTERLY MONITORING

EXTERNALEXPERTS ’ OPINIONS
GROUPS REPORT (QMR)

RESULTS
QUANTITATIVE DATA ON PROJECT MANAGEMENT MEETINGS (MM)
PIPELINE ANALYSIS (PA)
PROGRESS - WORKPLANS
MONITORING

IMPLEMENTERS’ PERIODIC
REVIEW OF PERIODICREPORTS REPORT APPROVAL PROCESS
REPORTS
ACTIVITIES

PRODUCTS DELIVERED BY REVIEW OF PRODUCTS AND PRODUCT REVIEW REPORTS (PRR)

CONTRACTORS SERVICES

MEANS &
RESOURCES
DIRECT OBSERVATION SITE VISITS SITE VISIT REPORT (SVR)
 Results chain in project planning

INPUTS
•Managemen
t OUTPUTS OUTCOMES IMPACT
•Expert •Project studied •Job created •Living condition of
•Equipment •Project executed •Income generation
people improved
•Fund •People trained •Project started •Health or
• material service educational status
•Time improved

Lesson learnt Feedback

Looking the project from sectoral and Local Government Development Strategy

LG ADMINISTRATION PROGRAM
– STRATEGIC CONTEXT

Effective , efficient
SECTOR PERSPECTIVE PROGRAM PERSPECTIVE Development of
& accountable
Urban sector
local gov .

Ethical Clear legal Building inst.

Stable LG Effective SME Human
standards - framework capacity in Infrastru cture
f inanc e system management development resources
focus on clients local gov

Results

LGs have LGs are

LGs have Participaiting units have resources to improve mgnt
ACCESS TO COMMITTED TO
RESOURCES
M GT TOOLS CHANGE Developed mgnt tools are accessible

Participating LG units are committed to change

FINANCE & Key Outputs

MGT TOOLS PERFORMANCE
TECHNOLOGY
developed INDICATORS Training of 4000 officials (600 LG units)
resources

Designing and testing ID Methodology

Database of management best practices

PERSONNEL is
COST OF MGT LEGAL
proactive and Recommendations to the legal framework
TOOLS FRAMEWORK
skill ful
Capacity and cooperation Ministry – LG Assoc.

Promoting ethical standards

Basis for performance benchmarking system

MGNT SUPPORT
PARTICIPATION
SYSTEM

INFO on
management tools
Reporting is an integral part of
monitoring and evaluation. Reporting is
the systematic and timely provision of
essential information at periodic
intervals.
Reporting includes:
▪ Physical works Activity
▪ Financial performance
▪ Time utilization
▪ Human resources
▪ Environmental issues
 M&E and the importance of images
M&E BY IMAGES WITHIN THE PROJECT/PROGRAMME CYCLE
Recommended
Phase M&E outputs Aim & use of information communication means
Appraisal Determine the need for change and the
Development rationale for the intervention Videos & photographs,
Sector specific appraisals Identify existing gaps and opportunities talking maps
for a potential development intervention
Planning Sub-System
Implementation Logical Framework Communicate change hypothesis and Infograph
results chain
Annual Operating Plans (AOPs) Clarify the yearly targets and activities Power-Point presentation
Databases Record data on quantitative & qualitative
indicators in an ongoing manner Videos, photographs &
Field visits Supervise, identify obstacles, negotiate, talking maps
build consensus, agree solutions
Evaluation Sub-System
Baseline Studies Determine the target population’s start Talking maps and
situation and context, as a benchmark for photographs, GIS, videos
future evaluations on conditions “prior to”
the intervention
Self-evaluations Determine the scope of results and why,
to inform the project strategy and create
Talking maps and
learnings for stakeholders photographs, GIS, videos
External evaluations Determine achievement of objectives and on conditions “following”
results based on effectiveness, relevance, the intervention, display
and sustainability criteria, etc. of results through Power-
Point presentation and
Systematisation Sub-System
infographs
Systematisations Reconstruct stakeholder processes and
experiences from a multiple perspective,
to create learnings and inform practice
Follow-up Evaluations Determine the net impact of
interventions, based on evaluation criteria
Systematisations Reconstruct processes and identify
learnings
 The formatted reporting system has simplified the task
 On time reporting has a value
 The importance of daily recording and documentation
 Importance of integration of departments
(municipality, OFED, revenue office, environmental
and land administration office, audit office)
 The importance of site visit and follow up
 From day to day , improvement in internalizing and
understanding the importance of M&E
 Integration and team spirit of all staff (specially staff
related to projects)
 Neatness and clarity of the information reduces the in and out turnover
of papers (reduces bureaucratic process)
 Little attention from general managers and experts.
 Increased concern of the senior regional, federal, WB and city officials
has alerted the project staffs to work hard.
Galatooma!