Adopted from Chris P.A. Bennett: ESSENTIAL FEATURES OF PROJECT PERFORMANCE ASSESSEMENT / MONITORING AND EVALUATION (M&E, MRV)

• Used to inform “lessons learned” and formulate recommendations
• As measured through four criteria (effectiveness, efficiency, sustainability, and impact) of Organic Compost project

Effectiveness: how well project was able to help targeted groups (Bennett, 2015)

Efficiency: whether benefits were delivered at lowest cost

Sustainability: whether benefits of project go beyond the project’s lifetime

Impact: whether there were consequences even beyond the direct consequences of the project

Example Summary of Findings for Organic Compost project (Team Ciherang)

Click image to expand.

Read more under Program Documents page. The following concepts are adopted from: Chris P.A. Bennett: ESSENTIAL FEATURES OF PROJECT PERFORMANCE ASSESSEMENT / MONITORING AND EVALUATION (M&E, MRV)

• Problem Tree (part of conceptual underpinning): looking at the core problem that the project is trying to solve and ensuring its implementation matches the root causes, not just the proximate causes of the core problem

• Investigating context: (potential intervening socio-economic and environmental factors and interactions) for purposes of evaluating project based on empiricism (has it worked elsewhere?), and beneficial impacts of a project

• Logical Framework: summary of a development project in matrix format outlining purpose, goals, activities and resources required, potential pitfalls, and ways of measuring achievement (Bond, 2003)
  • used to investigate the relevance of a project
  • determining whether implementation and design match project goals
  • based on core problem outlined in problem tree
  • identifies core components
  • encourages systematic thinking of a development project (Bond, 2003)
  • provides involved stakeholders with access to project details and an opportunity to make project decisions (Bond, 2003)

Example: Desa Ciherang Organic Compost Project Logical Framework (full evaluation available under Program Documents page)

• Key Stakeholder Analysis: Who is involved, which institutions, and how do they interact?. These questions can be used to assess whether interaction between stakeholders is sufficient, conflicts exist, decision-making is collective, and enough information-exchange is present.

• Identifying Assets: Done at both design and implementation phases, this step  involves asking important questions about which assets are needed and if there are sufficient ones, whether there is sustainability in attaining these assets, and equitable distribution of assets as well as proper management (e.g., physical, knowledge, human, financial, natural, and social) 

• Dialogue: analyzing stakeholder interaction and measuring the trajectory of trust (done at both design and implementation phases)

• Analyzing Governance: Done at implementation phase, this tool involves determining the status of institutional behavior on a project based whether it included many stakeholders, held decision makers accountable, etc.  

• Trajectory of trust: engaging all stakeholders, treating beneficiaries as equals, overcoming cultural barriers such as norms and hierarchies, pride, envisioning benefits for the whole society when empowering the rural poor, realizing the importance of information exchange (Bennett, Course Content/Notes)

• Appreciative inquiry: asking about general feelings on a project

• Cost benefit analysis: see Evaluations of Team Purwasari or Team Ciherang for examples under 2015 Evaluations page  

Challenges of villagers surrounding IPB Campus, rural West Java

Frequent dependency on external organizations/institutions

Challenges in:

• marketing agricultural products
• developing confidence
• managing the waste stream at the farm level
• ensuring high productivity
• maintaining sustainability of projects
• quality over quantity (taking on projects that will produce best results)

Challenges by project donors/implementers

 Challenges in creating appropriate assistance projects that might otherwise help achieve PMETR program goals:

• out of context (not considering urbanization, elites seizing benefits)
• inadequate (non- full cycle support/ no exit strategy)
• top down (little partnership, insufficient information exchange)
• emphasizing quantity over quality (trying to maximize rather than optimize)
• immeasurable (not recording trends, but rather looking at short-term status)

Photo Source: Maria Alejandra Cortes, UBC Graduate and 2015 Participant

The PMETR methodology begins with external evaluation when internal evaluation is non-existent or insufficient, eventually leading to self-sufficient internal evaluation and project implementation. The technique aims to build institutions, engage in partnerships across sectors, and measure indicators of effectiveness, sustainability, efficiency, and impact during design phase. It distinguishes clearly between the project design and implementation phases challenges and involves thorough evaluation of both (Bennett, 2015).

Click on image to expand.

The PMETR program consists of a two-fold process:
1) to evaluate the monitoring and evaluation technique itself as a tool for improving development practice and consulting
2) to evaluate the development project it is monitoring

Focuses on:
1) Institution building
2) Partnership

*Agriculture is just one of the many of project types PMETR can be used to evaluate

Utilizes:
• Relationship building
• Transparency
• Cultural Sensitivity

Adopts:
• More innovative development management approaches (e.g., outcome-based assistance, accounting for efficiency, participation, inclusivity, context, external factors)
• Specific features of successful projects to enable advancement and replication to build resilience and reduce poverty, broader goals of development (Bennett, Brief PPT 2+1)

Image Source: Maria Alejandra Cortes, UBC Graduate and 2015 Participant

The four-week program used rapid appraisal techniques (see Syllabus for more details):

• Week 1:
– Introduction of M&E and METR methodology using readings and lectures to teach tools (e.g. problem tree, logical framework, conceptual framework),
– Visits to locally- based international forestry research organizations employing M&E
– Visits to villages for rural Javanese cultural immersion
• Weeks 2 and 3:
– Separation into research teams (3-6 members of 3 core specializations: economics, governance, science/technology)
– Monitoring and data collection (e.g. stakeholder interviews, site observations, and village statistics) at an assigned village
• Week 4:
– Evaluation: gathering end findings to prepare a draft PowerPoint and report to be presented to project stakeholders and program staff at the partner university
– Receive feedback to finalize presentation and report

Welcome to UBC Blogs. This is your first post. Edit or delete it, then start blogging!