Step 6. Developing a monitoring and evaluation (M&E) plan
Rationale: why develop a monitoring and evaluation plan?
All projects need to carry out some form of evaluation: 1) to communicate to donors the expected and actual impacts of the project to help win funding to begin with, and then to show it was money well spent; 2) to show compliance with agreed work plan, and to negotiate changes to it when required; and 3) to provide systematic information to support learning and decision making during the implementation of the project. M&E of a project's impact pathways can provide all three.
Each line in the outcomes logic model table is an impact hypothesis. Each line hypothesizes that the project strategies will achieve the expected changes in KAS and practice of the respective actors. Reflecting on whether the hypotheses are valid and revising them helps the project learn as it goes along and identify necessary course corrections. This process of learning and adapting in real time based on data about how the project is unfolding is a practical way of doing adaptive management. Regular reflection on and the documentation of the validity of impact hypothesis constitutes action research into how project outputs and strategies do (and do not) lead to developmental outcomes in a specific context(s). Such monitoring and evaluation also provides the necessary information to write formal progress reports to donors, etc.
Preparation for the exercise:
You may want to introduce the idea and start the exercise in the workshop but it should be completed after the workshop, in consultation with project staff and key stakeholders who were not present.
How to develop a monitoring and evaluation plan:
Developing a monitoring and evaluation plan based on the outcomes logic model involves the following procedures.
Step 6.1 Identify outcome targets which are the main changes in KAS and practice that the project will work to achieve.
Step 6.2 Identify milestones that measure progress toward achieving the outcome targets.
Step 6.3 Design periodic reflection workshops to evaluate project progress and make course corrections.
Step 6.1 - Identify outcome targets
Identify the key outcomes that the project will work to achieve by the end of the project (Column 1, table below). These should be derived from the outcomes listed in the outcomes logic model. How you identify which outcomes the project will work to achieve is important. We recommend that it is done with participation of key project implementers and ideally with representatives of the key stakeholder groups (i.e., next users, end users and politically-important actors). Justification for prioritization should be given to key stakeholders, in particular those who might have attended the Impact Pathways Workshop but who are not part of the prioritization process.
Column 2 describes key assumptions--things that are beyond the control of the project but which affect project success. For example, a key assumption for a project working to improve product quality (e.g., fish, rice, etc.) is that farmers will receive a higher price for better quality.
In column 3, describe an outcome target that is SMART--specific, measurable, attributable, realistic and time-bound, for each outcome.
Column 4 descibes how the project will verify whether the outcome target has been achieved. This may be through an external evaluation or a project commissioned survey, etc. Also specify who will be responsible for verification and in what form the assessment will be presented.
The key outcomes—in terms of changes in KAS and Practice—that the project seeks to bring about (from Worksheet 3)
|
Assumptions
|
SMART outcome target
|
Means of verification?
By whom? In what form?
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Step 6.2 Identify milestones that measure progress toward achieving the outcome targets
You will find that most of the outcome targets are set at the end of the project. It is is useful therefore, to identify milestones to measure the project's progress towards these targets. We recommend that you reflect on progress along your impact pathways every six months (see step 6.3). Therefore, we recommend setting milestones for where you expect to be six months in the future when you hold the next reflection workshop.
Fill out the following table to identify SMART (specific, measurable, attributable, realistic and time-bound) milestones.
SMART Outcome Target (from Step 6.1)
|
SMART Milestone(s) to be achieved in the next six months (and beyond if you can)
|
Means of Verification? By whom? In what form?
|
Who is responsible for achieving the milestones?
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Step 6.3 Design periodic reflection workshops to evaluate project progress and make course corrections
You are most likely implementing your project in an uncertain environment. We expect that within the general scope of your work, as described in the project document, you will need to make course corrections to respond to opportunities and unforeseen circumstances. Indeed, proactively looking for opportunities is a characteristic of projects that have achieved developmental impacts.
You need a process to spot opportunities and threats, and to test your impact hypotheses described in the outcomes logic model. The process we recommend is that every six months you convene a ‘reflection workshop’. This may be done as a separate workshop, part of a scheduled project meeting or a virtual meeting by telephone or Skype (i.e. VOIP). What is key is that there be time and a space made available for project team members, and preferably also key stakehold, to reflect on progress-to-date.
This reflection process is also shown graphically here.
1. During the PIPA workshop, participants develop a shared view of where they want to be in two years’ time, and describe impact pathways to achieve that vision. The project then implements strategies, which lead to changes in KAS and practice of the participants involved.
2. A workshop is held six months later to reflect on progress. The vision is changed to some extent, based on what has been learnt, the outcome hypotheses are revised when necessary and corresponding changes are made to project activities and strategies. New milestones are set for the next workshop.
3. The process continues. The project never achieves its vision (visions are generally used to motivate and stretch), but it does make real improvements.
Click here to see a Suggested reflection workshop agenda.
These reflections are the culmination of one set of experiential learning cycles and the beginning of others. If the reflections are well documented, they can be analyzed at the end of the project to provide insights into how interventions do, or do not, achieve developmental outcomes in different contexts. PIPA M&E thus provides a framework for carrying out action research. The quality of the research depends on the facilitation of the reflections, the data used and the documentation of the process. PIPA M&E is not prescriptive about the data used in the reflections, but does encourage researchers to gather data using multiple methods. It also recommends ways of introducing thematic and gender perspectives into the design of data-gathering methods and reflection processes.
Comments (0)
You don't have permission to comment on this page.