Rainbow Evaluation
Framework
Better Evaluation
(Reading)
Rainbow Evaluation Framework
• A framework prompts you to think about a series of key
questions.
• Components of Rainbow Framework are as under:
– Manage
– Define
– Frame
– Describe
– Understand Causes
– Synthesise
– Report and Support Use
Rainbow Evaluation Framework
• Framework is organized around 7 clusters of evaluation tasks
• Four of the clusters relate to methods and processes for
planning and implementing the evaluation:
– Manage
– Define
– Frame
– Report and support use
• Three of the clusters group methods and processes to answer
different types of key evaluation questions:
– Describe
– Understand causes
– Synthesize data
Rainbow Evaluation Framework
Manage Define Frame Report & Support Use
Describe
Understand Causes Synthesise
Key Evaluation Questions
• These are high level questions that the evaluation is intended to
answer, for example, "How effective was the program?"
• KEQs usually should be maximum 5 – 7 main questions and need
to be developed and agreed on at the beginning of evaluation
planning
• Key Evaluation Questions should be developed by considering
the type of evaluation being done, its intended users, its
intended uses (purposes), and the evaluative criteria being used.
Key Evaluation Questions
Type Typical key evaluation questions
Process How is the program being implemented?
evaluation How appropriate is the processes compared with quality standards?
Is the program being implemented correctly?
Are participants being reached as intended?
How satisfied are program clients?
Outcome How well did the program work?
or Impact Did the program produce or contribute to the intended outcomes in
evaluation the short, medium and long term?
To what extent can changes be attributed to the program?
What were the features of the program that made a difference?
What was the influence of other factors?
Key Evaluation Questions
Type Typical key evaluation questions
Economic Has the intervention been cost-effective (compared to
evaluation alternatives)?
(cost-effectiveness Is the program the best use of resources?
analysis and
cost-benefit
analysis)
Rainbow Evaluation Framework - Manage
Rainbow Evaluation Framework
• Manage an evaluation; Manage an evaluation including
deciding who will conduct the evaluation and who will make
decisions about it:
– Understand and engage stakeholders: Who needs to be
involved in the evaluation? How can they be identified
and engaged?
– Establish decision making processes: Who will have the
authority to make what type of decisions about the
evaluation? Who will provide advice or make
recommendations about the evaluation?
– Decide who will conduct the evaluation: Who will
actually undertake the evaluation?
Rainbow Evaluation Framework
• Manage an evaluation
– Determine and secure resources: What resources (time,
money, and expertise) will be needed for the evaluation
and how can they be obtained? Consider both internal
(e.g. staff time) and external (e.g. previous participants’
time) resources.
– Define evaluation standards: What will be considered a
high quality evaluation?
– Document management processes and agreements: How
will the management processes and agreements be
documented?
Rainbow Evaluation Framework
• Manage an evaluation
– Develop planning documents for evaluation: What needs
to be done to design, plan and implement the
evaluation? What planning documents need to be
created (evaluation framework, evaluation plan,
evaluation design)?
– Review evaluation (do meta-evaluation): How will the
evaluation itself be evaluated including the plan, process,
and report?
– Develop evaluation capacity: How can the ability of
individuals, groups and organizations to conduct and use
evaluations be strengthened?
Rainbow Evaluation Framework
• Define what is to be evaluated; Define what is to be
evaluated and how it is understood to work:
– Develop initial description: What exactly is being
evaluated?
– Develop program theory / logic model: How is the
intervention understood to work (program theory,
theory of change, logic model)?
– Identify potential unintended results: What are possible
unintended results (both positive and negative) that will
be important to address in the evaluation?
Rainbow Evaluation Framework
• Frame the boundaries of an evaluation; Set the parameters
of the evaluation – its purposes, key evaluation questions
and the criteria and standards to be used:
– Identify primary intended users: Who are the primary
intended users of this evaluation?
– Decide purpose: What are the primary purposes and
intended uses of the evaluation?
– Specify the key evaluation questions: What are the high
level questions the evaluation will seek to answer? How
can these be developed?
– Determine what ‘success’ looks like: What should be the
criteria and standards for judging performance?
Rainbow Evaluation Framework
• Describe activities, outcomes, impacts and context; Collect data
to answer questions about program activities and the context in
which it has been implemented:
– Sample: What sampling strategies will be used?
– Use measures, indicators or metrics: What indicators will be
used? The existing ones or need to develop new indicators?
– Collect data: How will you collect data about activities,
results, context and other factors?
– Manage data: How will you organize and store data?
– Combine qualitative and quantitative data: How will you
combine qualitative and quantitative data?
– Analyze data: How will you analyze data?
– Visualize data: How will you display data visually?
Rainbow Evaluation Framework
• Understand Causes of Outcomes and Impacts; Collect and
analyze data to answer causal questions about what has
produced outcomes and impacts that have been observed:
– Check the results support causal attribution: How will you
assess whether the results are consistent with the theory?
– Compare results to the counterfactual: How will you compare
the factual with the counterfactual - what would have
happened without the intervention?
– Investigate possible alternative explanations: How will you
investigate alternative explanations?
Rainbow Evaluation Framework
• Synthesize data; Combine data to form an overall assessment of
the merit or worth of the intervention, or to summarize evidence
across several evaluations:
– Synthesize data from a single evaluation: How will you
synthesize data from a single evaluation?
– Synthesize data across evaluations: Do you need to synthesize
data across evaluations?
– Generalize findings: How can the findings from this evaluation
be generalized to the future, to other sites and to other
programs?
Rainbow Evaluation Framework
• Report and Support Use; Develop findings in ways that are useful
for intended users, and support them to make use of them:
– Identify reporting requirements: What timeframe is required
for reporting?
– Develop reporting media: What types of reporting formats
will be appropriate for the intended users?
– Ensure accessibility: How can the report be easy to access and
use by different users?
– Develop recommendations: Will the evaluation include
recommendations? How to develop and by whom?
– Support use: In addition to engaging intended users, how will
you support the use of evaluation findings?
