Terms of Reference

External evaluation

Education in Emergency Response Project – Endline Evaluation

INDEX

1. INTRODUCTION.. 3

1.1. Background and rationale. 3

1.2. Purpose of the evaluation. 3

1. PRESENTATION OF THE ACTION TO BE EVALUATED.. 3
2. EVALUATION QUESTIONS AND CRITERIAS.. 3
3. WORK APPROACH AND METHODOLOGY.. 8
4. STRUCTURE AND FORMAT OF THE EVALUATION REPORT. 8
5. EVALUATION TEAM.. 9
6. PRESENTATION OF THE TECHNICAL AND ECONOMIC OFFER AND EVALUATION CRITERIA.. 10

1. INTRODUCTION

1.1. Background and rationale

Founded on the Gospel, the mission of Secours Catholique Caritas France is to offer love and awareness of solidarity in France and around the world. Secours Catholique invites everyone to engage with, support others, and experience the joy of fraternity. Its members work to increase everyone’s ability to act so that people can live in dignity. Secours Catholique-Caritas France fights the causes of poverty, inequality and exclusion and proposes alternatives that serve the common good.

Partner background

Trocaire has been providing humanitarian and development assistance in the Gedo region of Southern Somalia since 1992. Gedo is the second largest region in the country, home to over half a million people and 168,000 internally displaced persons (IDPs). Our long- established presence and commitment to the people in the region has resulted largely from our close partnership with the communities in which we work and has earned us the trust and respect of the local population and elders of the community. These relationships have enabled us to continue operations in Gedo, despite the prevailing insecurity, and have been the foundation on which we continue to reach the most vulnerable, working towards bringing about positive and sustainable change.

1.2. Purpose of the evaluation

The external evaluation purpose is to assess the impact of the Education in Emergency Response Project in the region, on the local community, stakeholders and Trocaire. To draw out lessons learned with the broader aim of improving policies and practices, and enhancing accountability.

2. PRESENTATION OF THE ACTION TO BE EVALUATED

General Objective: **

To provide sustained access to quality education and lifesaving risk reduction information to school communities in the effort to mitigate the impact of COVID 19 pandemic in Gedo Region of Somalia.

Specific Objective:

• School-going children in Luuq, Belet Hawo and Dollow districts have sustained access to quality education;
• Pupils and the school community in Gedo are at reduced risk from COVID-19;
• Enhance the status of women and girls through promotion of gender equality
• Institutions that are responsible for guaranteeing sustained access to quality education are strengthened to better respond to education needs within target locations.

Long-Term Outcome 1:

School-going children in Luuq, Belet Hawo and Dollow districts have sustained access to quality education;

v Intermediate outcome 1.1: School going children have access to a conducive learning environment

Long- Term Outcome 2:

Pupils and the school community in Gedo are at reduced risk from COVID-19;

v Intermediate outcome 2.1: Pupils and teachers have knowledge and access to COVID-19 prevention measures

v Intermediate outcome 2.2: Personal hygiene is promoted in all seven target schools;

v Intermediate Outcome 2.3: Improved wellbeing of children and parents/ caregivers;

v Intermediate Outcome 2.4: CECs and children demonstrate improved skills on food production

Long-Term Outcome 3:

Institutions that are responsible for guaranteeing sustained access to quality education are strengthened to better respond to education needs within target locations.

v Intermediate outcome 3.1: Local institutions responsible for ensuring provision of education services are strengthened to better deliver on their mandate.

3. EVALUATION QUESTIONS AND CRITERIAS

An evaluation of the project is proposed according to the following evaluation framework, in which the key questions will be analysed in accordance with the following criteria, as well as any other aspects deemed relevant in the process of obtaining information and analyzing it.

The criteria to be used will be the 6 OECD Development Assistance Committee (DAC) criteria which was updated in 2019[1] and a seventh criterion added to measure accountability.

The key questions to be used per criteria are taken from the Core Humanitarian Standard[2].

Criteria

Key Questions (by CHS Pillar – Core Humanitarian Standard)

Relevance

The extent to which the intervention objectives and design respond to beneficiaries’,5 global, country, and partner/institution needs, policies, and priorities, and continue to do so if circumstances change.

Pillar 1: Humanitarian response is appropriate and relevant.

1. Was the intervention design and implementation based on an unbiased assessment of needs and risks, as well as an assessment of the vulnerabilities and capacities of the different groups?
2. Was the intervention adapted to evolving needs, capacities, risks and context?
3. How have vulnerable groups been identified?
4. Does the response include different types of assistance and/or protection for different demographic groups?

Pillar 2: Humanitarian response is effective and timely

1. Are constraints and risks regularly identified and analysed, and plans adapted accordingly?

Pillar 3: Does the humanitarian response avoids negative effects?

1. Were the results of the community risk assessment and preparedness plan used to guide activities?
2. To what extent does the response strategy anticipate the risk of negative effects (“do no harm” approach)?
3. In what ways (both formal and informal) are local leaders and/or authorities consulted to ensure response strategies are in line with local and/or national priorities?

Coherence

The compatibility of the intervention with other interventions in a country, sector or institution

General

1. Is there internal coherence? (Synergies and interdependencies between the interventions carried out by the NGO, as well as coherence between the intervention and the international norms and standards to which the NGO adheres).
2. Is there external coherence (coherence between the intervention under consideration and those carried out by other actors in the same context). It encompasses complementarity, harmonization and coordination with other actors, and verifies that the intervention brings added value while avoiding duplication of activities.

Pillar 4: Humanitarian response is effective and timely

1. Have unmet needs been reported to organizations with the relevant expertise and mandate, or has advocacy been conducted to ensure that these needs are addressed?

Pillar 6 : Humanitarian response is coordinated and complementary.

1. Are the programmes of other organisations and authorities taken into account when designing, planning and implementing programmes?
2. Are gaps in coverage identified and addressed?
3. Has participation in relevant coordination structures and collaboration with other organizations been ensured so as to minimize demands on communities and maximize coverage and service delivery of the overall humanitarian response?
4. Has the necessary information been shared with partners, coordinating groups and other relevant actors using appropriate communication channels?

Effectiveness

The extent to which the intervention achieved, or is expected to achieve, its objectives, and its results, including any differential results across groups.

General

1. Was the specific objective correctly formulated and was it achieved?
2. Did the planned indicators allow to measure the achievement of the specific objective? Were they disaggregated by gender, age or other vulnerabilities?
3. Are the sources of verification reliable, sound, and reasonably priced?

Pillar 2: Humanitarian response is effective and timely

1. Does planning consider optimal times for activities, accounting for factors such as climatic factors, season or conflict?
2. Are early warning systems and contingency plans used?
3. Have relevant technical standards and good practices used in the humanitarian sector been used?
4. Is a program monitoring system used to adjust programs and address quality of work issues?

Pillar 3: Does the humanitarian response avoids negative effects?

1. To what extent does the response strategy mitigate the risk of negative effects (“do no harm” approach)?

Efficiency

The extent to which the intervention delivers, or is likely to deliver, results in an economic and timely way.

General

1. Were the expected results and their indicators properly formulated and were they achieved? Were they performance indicators rather than activity indicators? Were they disaggregated by gender, age, or other vulnerabilities?

Pillar 8: Staff are supported to do their job effectively, and are treated fairly and equitably

1. Is staff performance managed, under-performance addressed and good performance recognised?
2. Are staff aware of support available for developing the competences required by their role and are they making use of it?
3. Do all staff have updated job descriptions and objectives, including specific responsibilities and objectives?
4. Was the composition of the project teams adequate to ensure the participation of all vulnerable groups? (Women, the elderly…)
5. Has the implementing partner put in place a system to ensure the well-being of the project teams?

Pillar 9: Resources are managed and used responsibly for their intended purpose

1. Have programs been designed and processes implemented to ensure the efficient use of resources, balancing quality, cost and timeliness of each phase of the intervention?
2. Have resources been managed and used to achieve the intended objective, minimizing waste?
3. Is expenditure monitored regularly and the reports shared across programme management?
4. Have the risks of corruption been managed and have appropriate measures been taken if such risks have been identified?
5. Are services and goods procured using a competitive bidding process?
6. Are potential impacts on the environment (water, soil, air, biodiversity) monitored, and actions taken to mitigate them?

Impact

The extent to which the intervention has generated or is expected to generate significant positive or negative, intended or unintended, higher-level effects.

General

1. What are the social, environmental and economic effects over the long term or on a large scale?
2. What are the potential effects on people’s well-being, human rights, gender equality and the environment?

Pillar 7: Humanitarian actors continuously learn and improve.

1. Were lessons learned and past experiences used to design the programs?
2. Did learning, innovation and implementation of change come from monitoring and evaluation, as well as feedback and complaints?
3. As the learning and innovation been shared internally, with communities and individuals affected by the crisis, and with other stakeholders?

Sustainability

The extent to which the net benefits of the intervention continue, or are likely to continue.

Pillar 3: Humanitarian response strengthens local capacities?

1. Has a transition or exit strategy been planned from the early stages of the humanitarian program to ensure longer-term positive effects and reduce the risk of dependency?
2. Were programs based on local capacities and designed to enhance the resilience of communities and individuals affected by the crisis?
3. Did the intervention contribute to the development of local leadership and local organizations in terms of their capacity as first responders in the event of future crises, while taking steps to ensure that marginalized and disadvantaged groups are adequately represented?
4. Resilience and Humanitarian-Development-Peace Nexus approach: have complementarities between humanitarian and development actors been established?
5. Climate change/environment**: did the project strategy include minimizing the carbon footprint and increasing the climate resilience of the humanitarian response?

Accountability

Pillar 4: Humanitarian response is based on communication, participation and feedback

1. Has information been provided to communities and individuals affected by the crisis about the organization, the principles it upholds, the behaviour it asks of its staff, the programs it implements, and the assistance it seeks to provide?
2. Has this information been provided in languages, formats and media that are easily understood, respectful and culturally appropriate for different members of the community, especially vulnerable and marginalized groups? (Women, the elderly, etc.).
3. Has the inclusiveness of representation been ensured, involving the participation and commitment of communities and people affected by the crisis in all phases of the intervention?
4. Are equitable opportunities promoted for participation of all groups in the affected population, especially marginalised and vulnerable people?
5. Have communities and people affected by the crisis been encouraged and supported to report their level of satisfaction with the quality and effectiveness of the aid received, paying particular attention to the gender, age and diversity of those giving their views?

Pillar 5: Complaints are welcomed and addressed

1. Have communities and individuals affected by the crisis been consulted on the design, implementation and monitoring of the complaints processes?
2. Was there communication about how the mechanism was accessible and the scope of issues it could address?
3. Is the complaints handling process documented and in operation? This process should cover the program, sexual exploitation and abuse, and other forms of abuse of authority.
4. Have complaints been handled in a timely, fair and appropriate manner that prioritizes the safety of the complainant and those affected at all stages?
5. Are there agreed and respected timeframes to investigate and resolve complaints? Is the time between a complaint is filed and its resolution recorded?
6. Are there specific policies and procedures in place to deal with situations of sexual exploitation, abuse or discrimination? Are they known to staff?

Pillar 8: Staff are supported to do their job effectively, and are treated fairly and equitably

1. Do staff sign a code of conduct or similarly binding document? If so, do they receive orientation on this and other relevant policies which allows them to understand it properly?
2. Are complaints received about staff or partners’ staff? How are they handled?
3. Are suppliers asked to sign a code of conduct (including the prevention of sexual exploitation and abuse) and is this code of conduct appropriately presented to them?

4. WORK APPROACH AND METHODOLOGY

The evaluation team will carry out a methodological proposal for the achievement of the objectives set in this evaluation. This proposal will be sent to Secours Catholique-Caritas France and to Trocaire Somalia for validation before the beginning of the field work. The work plan will include a cabinet phase and a fieldwork phase.

The evaluation will take an LoE of 21 days.

The consultant will be responsible for defining and implementing the overall approach of the evaluation. The consultant’s work includes defining data collection and analysis techniques, structured field visits, and interactions with beneficiaries and the evaluation team. The tools, methodology and conclusions of the evaluation must be reviewed and validated with various stakeholders and approved by the person in charge of the evaluation at Secours Catholique.

The results of the evaluation must be presented, distinguishing between: data, interpretations and value judgments. The conclusions will be presented, as well as the lessons learned (cause and effect relationship between the activities carried out and the conclusions obtained) and the recommendations (proposal to improve the cause and effect relationship and the logic of the intervention, information systems that are recommended to be put in place, etc.).

5. STRUCTURE AND FORMAT OF THE EVALUATION REPORT

The evaluation team will be asked to provide:

• An interim report (electronic version in Word format). Concluding the evaluation fieldwork, this report will be submitted prior to the return within a reasonable period of time to allow the interested parties to familiarize themselves with all the documents.
• A final report (in electronic version in Word format and on paper between 30-50 pages max, not including annexes). The latter will be accompanied by an executive summary of 3-4 pages maximum including the essential information of the report. The report will integrate the remarks made during the restitution meeting, this final report will be produced in 3 paper copies AND sent in electronic version.

The evaluation report should contain the following index:

1. Executive Summary: (3-4 pages maximum): including main conclusions and recommendations.
2. Introduction:

ü Background and purpose of the evaluation.

ü Initial Questions and Criteria: Definition.

1. Summary description of the intervention evaluated, summary of the background, organization and management, actors involved and context in which the intervention takes place.
2. Methodology used in the evaluation.

ü Methodology and techniques applied

ü Prerequisites and limitations of the study carried out.

1. Analysis of the information collected and answers to the key questions of the evaluation criteria.
2. Conclusions of the evaluation, in relation to the established evaluation criteria.
3. Lessons learned from the overall conclusions that illustrate good practice and can feed back into the intervention’s actions or be used for future interventions.
4. Recommendations classified according to the criteria chosen by the evaluation team. If possible, mention the actors to whom the recommendation is addressed.
5. Appendices:

ü ToRs,

ü The work plan, composition and description of the mission.

ü Proposed methodology, techniques and sources used to collect information.

ü Literature review: list of secondary sources used.

ü Interviews: list of informants, interview plans, transcripts and notes.

ü Surveys: models, raw data collected and statistical analysis.

ü Participatory workshops: report and products.

ü Claims and comments from different stakeholders on the draft report if they are relevant, including any disagreements that were not reflected in the report.

6. EVALUATION TEAM

The evaluation team will be composed of one or two experts who should correspond to the desired profile:

• Master’s degree (or assimilated disciplines),
• More than 7 years of professional experience with at least 5 years of experience in managing and conducting project evaluations,
• Knowledge of results-based management principles,
• Knowledge of the Core Humanitarian Standard,
• Mastery of participatory techniques and other approaches to data collection and analysis,
• Ability to put forward group dynamics, organize and propose discussion workshops and prepare capitalization documents.

The evaluation team will have to present the curriculum vitae of each of its members.

The evaluation team will have to appoint an evaluation coordinator who will have final responsibility for the work and who will serve as a contact with the evaluation management unit.

The evaluation will be carried out in (country) in close collaboration and with the participation and support of Trocaire as well as with the active participation of the stakeholders identified in point 2.

7. PRESENTATION OF THE TECHNICAL AND ECONOMIC OFFER AND EVALUATION CRITERIA

The technical proposals will have to respect the following characteristics :

• Cover page:

§ Name of the firm or consultant

§ Contact information for the firm or consultant

§ References, CV and experience of the consultant(s),

§ Where applicable, the specific roles and responsibilities of the team leader, supervisory chain, and other key members of the evaluation team.

• Technical offer: indicating understanding of the ToR, the selected evaluation methodology, the evaluation matrix and a detailed work plan.
• Financial offer: including the overall budget (excluding tax and all taxes if VAT is applicable) and detailed prices (fees, per diems, transportation, etc.).

Bids received will be evaluated according to the following criteria and scales:

CRITERION 1: Profile and experience of evaluators (40%)

CRITERION 3: Methodological proposal (40%)

CRITERION 4: Economic offer (20%).

Bidders must also indicate the country of which they are a national by presenting the usual proof in this respect according to their national law (registration number + copy of passport or national identity card).