Final Evaluation for the Education in Emergency Project in Galkacyo South District.

Evaluation of the Project: Education in Emergency Project for Most Vulnerable Internally Displaced Persons (IDPs) in Galkacyo South District
Project code: K-SOM-2021-4070
Commissioned by: Diakonie Katastrophenhilfe -DKH supporting the local implementing partner Centre for Peace and Democracy (CPD).

A. DKH and CPD Overview.

Diakonie Katastrophenhilfe (hereafter DKH) is a humanitarian assistance agency in Germany with headquarters in Berlin. It has regional offices in Asia, Africa, and Latin America, as well as country offices in our priority countries, which are responsible for the coordination of aid programs. DKH’s operating model is structured around implementing projects exclusively through a network of local partner organizations. As such, DKH’s core business is partner support and partner strengthening.
On average, DKH supports around 150 projects annually across the globe. Besides humanitarian assistance, DKH also focuses on Disaster Risk Reduction (DRR), preparedness measures as well as rehabilitation and reconstruction. In recent years, DKH has accelerated its commitment and investment in strengthening its cash and voucher assistance (CVA) approach to ensure context-appropriate and dignified responses for affected families.
DKH’s projects are funded through donations, a percentage of the annual income of sister agency ‘Bread for the World’, collaborations with other aid agencies such as Caritas International, as well as institutional donor funds such as the German Federal Foreign Office (GFFO) and the Federal Ministry for Economic Cooperation and Development (BMZ). DKH is also a recognized cooperation partner of the ECHO, the Office for Humanitarian Aid of the European Commission, as well as EuronAid, a network of European non-governmental organizations specialized in food aid and food security.
In Somalia, DKH has been running intensive relief aid programs for more than twenty years. In recent years, DKH and its local partner organizations implemented also transitional aid programs. In close cooperation with its local partner organizations, DKH works across several sectors including WASH, Shelter, Camp Management and non-food Items, Education, Food Security, Livelihoods with integrated Cash Voucher Assistance, and livelihoods and vocational training. wherever feasible.
Centre for Peace and Democracy (CPD), a local Non-Governmental Organization working in Somalia has been implementing the “Education in Emergency (EiE) Project in Galkacyo South District” with funding from the DKH Somalia program. Therefore, South Galkacyo is one of the IDP hosting Districts in Somalia with an estimated 45 IDP Camps (population of IDPs estimated at 9,364 Households (55,528 people) according to CCCM Report for Galkacyo South.
B. The project Background.
The Education in Emergency (EIE) project supports around 1,200 (50%F) school-aged children from at least 800 vulnerable IDP households located in various IDP settlements in the Galkacyo South District. The basic approach of the project is designed to address the children, girls, and boys, in need of education, directly through an individual selection of the most vulnerable children that are forcefully displaced from their families or separated from their families and live now in foster households.
Education provides immediate physical, psychosocial, and cognitive protection during crises. Out-of-school children are more vulnerable and exposed to protection risks like; GBV, early marriages, early pregnancy, child labor, and forced recruitment. Opposed to out-of-school children, children with a certain level of education are less vulnerable to illicit recruitment, unwanted pregnancy, and early marriage. Educated youths, girls and boys, have better chances to find employment and social integration.
The project supported school-aged children including children living with disability (50% girls, 50% boys) in ages between 6 and 13 years for primary school learning level (~95% of beneficiaries), and ages 14 to 18 for secondary school level education (whereby in single cases those age limits may vary due to the generally late enrollment of children in school) selected from vulnerable IDP households.
The project focused on children for new enrolments for small children, girls and boys, who are not enrolled in any primary school and re-enrolment for those that have dropped out of school due to their background of displacement and related vulnerability and poverty constraints, those usually in a higher age and in higher classes, and are willing to attend school, as well as retention in classes for vulnerable secondary school girls and boys that already have dropped out or that are in danger to drop out shortly before their final exams due to vulnerability constraints of their households and their inability to pay school fees and other ancillary costs attached to their education.
C. Purpose of the Final Evaluation
The purpose of this final evaluation is to establish whether the activities lead to the achievement of the planned results as well as the project objectives. The evaluation will include a rigorous analysis of the project design, participation of key project stakeholders, administration and management of resources, and the impact on the beneficiaries (direct and indirect). The evaluation will inform the DKH, and the project partners of key lessons learned, and propose recommendations for future design and implementation of similar projects. Specifically, the final evaluation will focus on the following aspects:
• To inform DKH/CPD and other project stakeholders on the progress in implementation of the EIE project, i.e. the delivery of planned Objectives and Results achieved as well as progress towards achievements of the respective objectives while highlighting enabling factors and challenges encountered.
• Drawing key findings and lessons learned and proposing improvement and adjustment needed to ensure that the project remains relevant to the needs of the target groups and contributes to knowledge development.
• To identify the factors (positive and negative) that have impacted the achievement of the objectives and expected results within the project time frame.
• To analyze what is working well for the project, the main bottlenecks, and what needs to be done in the future for similar projects.
• To inform resource allocation for a future round of proposals, as well as to inform decisions on scaling-up, replication, and /or continuation.

D. Project Objectives
The Overall project objective: Enhanced resilience of crisis-affected IDP boys and girls, especially out-of-school children in the South Galkacyo district in Galmudug State, Somalia.
Specific objective: Improved enrollment and retention of qualitative education in emergencies for girls and boys.
The Specific Objective indicators
• 75 % of targeted school-aged children are constantly in schools in the IDP camps and attend classes regular basis.
• 75 % of new and current enrolled learners (50% F and 50% M) are retained and complete at least one school term during the project period.

Result 1. Ensure emergency and crisis-affected children have access to a safe and protective learning environment
Indicators for Result 1
• 1200 children (600 M/ 600 F) reached with Education in Emergency assistance.
• 25 classrooms constructed or rehabilitated in 10 IDP schools.
• 11 child-friendly and inclusive WASH facilities are available to emergency-affected children and youth in 6 of the 10 targeted schools.

Result 2. Ensure targeted most vulnerable children, respective HHs, and teachers’ basic needs are adequately addressed as a key barrier to enrolment and retention in education.
Indicators for Result 2
• 30 teachers (21 M/ 9 F) benefitting from vouchers for school fees to pay emergency incentives
• 1.200 learners-beneficiaries (600 M/ 600 F) with access to food rations through monthly food vouchers.

Result 3. Strengthen the capacity to deliver effective and coordinated education in emergency preparedness and response within the education system.
Indicators for Result 3
• 30 teachers (21 M; 9 F) trained in different modules according to the teacher in Crisis Training Module.
• 70 Community Education Committee members (50 M, 20F) trained in school safety, various TiCC modules, and hygiene promotion in schools, as well as provided advocacy campaigns on the importance of education, especially for girls in Galkayo South.

E. Scope of the Final Evaluation
The final evaluation will cover the assessment of the key activities implemented during the duration of the project in the time frame between November 1st, 2021, and October 31st, 2023. The evaluation will target the 10 schools in the 10 IDP camps; covering the main areas of intervention and evaluating the main results concerning the indicators mentioned in the project objectives section D above, considering the impact of gender and inclusion of the project as of the end of the project period.
F. Approach and Methodology
Approach
The proposed final evaluation seeks to provide plausible evidence to that will be used for learning, inform future designs of education projects in similar contexts, showcase best practices, and provide accountability on resources.
The evaluation will be highly participatory and shall involve engagement with the EIE project stakeholders involved in the project. These shall include the Galmudug State Ministry of Education, Galkayo South District education department, Community Education Committees (CECs), IDP Camp Management Committees, Parents of the learners, teachers, and participating EIE project schools’ management.
The identified EIE stakeholders will be involved in providing their roles, experience, and perception of the various EIE project activities, identifying issues, challenges, and the solutions used to address the issues.The approach shall also be required to incorporate the analysis of records to ascertain the actual performance of the parameters of EIE indicators including enrolment, retention, and improvement in teaching skills class performance based on administered tests.
Methodology
The consultant shall be required to develop a viable evaluation methodology that is based on the Education in Emergency evaluation parameters. The methodology and tools used will be required to capture, analyze, and rate the following: – I). Improvement in access to education for the emergency-affected populations targeted IDP children, II). Provision of learning space including improvement of the learning environment through the construction of temporary learning spaces (TLS), III). Provision of Vouchers (conditional and flexible) for Education (V4E) to cater to their primary and/or secondary education that will be redeemable by the school administration in EIE project selected schools to cover teacher incentives, teaching materials, school maintenance costs, and basic supply of clean water to the learners for drinking and handwashing.
The evaluation will be required to describe the study design and a detailed sampling methodology sources of information used (documents, respondents, administrative data, literature, etc.) in sufficient detail so that the adequacy of the information can be assessed. The evaluation methodology shall be required to explain the selection of case studies or any samples used in the evaluation and their usefulness. The methodology used shall also develop viable tools and processes to help capture data and information that will contribute to analyzing the achievement of the processes used in delivering the project and also the degree to which the indicators have been achieved. The methodology shall provide the probable limitations regarding the representativeness of the samples identified and alternative/mitigation measures to be adopted. The evaluation methodology proposed by the consultant shall be capable of undertaking cross-validation of the information sources and critically assessing the validity and reliability of the data. The consultant shall also be required to provide and include a complete list of interviewees and other information sources consulted, to the extent that this does not conflict with the privacy and confidentiality of participants.
The consultant will be required to select/develop and adapt the data collection tools that will be relevant to use in the specific context of this assignment as provided in the scope of work. The sample tools will be required to conform to the approach recommended by the Somalia Education Cluster relevant to the assessment of needs in emergencies. An appropriate and strategic mixed sampling method should be indicated and its potential to provide the data and information required indicated/justified.
The suggested methodology will include:
a) An in-depth desk study analysis and mapping of all project documents, relevant material, and resources, including:
• The project proposal
• The project log frame
• Vulnerability assessment report
• Education in the emergency assessment report
• Project narrative progress reports generated up to the mid-point of project implementation.
• Project financial reports.

b) Questionnaires and surveys were addressed to appropriate groups of stakeholders. Semi-structured Interviews, focus group discussions, and meetings (in person, via Skype and telephone) with relevant DKH & CPD staff, donor representatives, and various stakeholders; – to gather substantial anecdotal evidence on the effectiveness, efficiency, relevance, and timeliness of the project activities implementation and delivery.

c) Selected field visits to meet with the local stakeholders, beneficiaries, and partners involved in the implementation of the EiE project, including interviews and focused group discussions with direct beneficiaries to determine the relevance and effectiveness of the project.

d) Inception Workshops to evaluate the evaluation, methodology, design, and sample.
e) A Final evaluation presentation to present and discuss findings and recommendations.

G. Key Guiding Questions
The consultant will be required to develop a methodology that is in line with the guidelines of the OECD evaluation criteria of relevance, effectiveness, efficiency, coherence and sustainability, and impact.
Based on the EIE project’s design and implementation as undertaken so far, the following key questions have been developed to guide the development of the assignment approach, and methodology and inform the final findings.
a) Relevance
• To what extent are the projects, activities, and outputs aligned with the project goals as captured in the proposal?
• Was the project designed in a way to meet the educational needs of targeted beneficiaries?
• Has the project successfully prioritized the most vulnerable IDP children as envisioned in the project design?
• Has the project succeeded in creating a safe and protective learning environment for the learners enrolled in schools by the project? If not, why and what it needs to be done?
• Has the project been successful in ensuring at least 75% of the enrolled beneficiaries are consistently attending school? If not, why and what needs to be done?
• Has the project ensured that at least 75 % of new and current enrolled learners (50% F and 50% M) are retained and complete at least one school term during the period covered by the evaluation?
• Has the project improved teachers’ and community education committees’ capacity to support quality education access in emergencies for IDP children?
• How successful has the project been in ensuring the inclusion of girls compared to boys?
• What is the perception of the community and relevant stakeholders towards this project and the impact it has on the community?
• What has been the impact of external factors such as COVID-19, climate change, and conflict on the implementation of the project against the activity plan?
• How did the proposal conform to the priorities in emergency education as outlined by the Somalia Humanitarian Response Plan and other Galmudug State education strategies and plan?

b) Effectiveness and Efficiency
• What have been the key outputs of the projects?
• Is output delivery on schedule? Are there any delays? What have been the main challenges in the delivery of these outputs?
• Which activities and use of the produced output were the most/least effective in contributing to the projects’ objectives and why? What are the common quality aspects and challenges?
• What are the main factors that have facilitated or obstructed the achievement of outcomes?
• How well has the project managed partnerships with the school management, Department of Education, and local government units? How did the quality of partnerships affect the delivery of the project outputs?
• Were there adequate monitoring tools and mechanisms in place, and functional? Do they allow communication and exchange of lessons learned?
• Have the existing monitoring and evaluation mechanisms contributed to the achievement of expected results throughout the project implementation and how have these been effective? What adjustments did the project make to the monitoring mechanism and what difference did this make?
• Has the project been implemented in a cost-efficient manner?
• Have other unforeseen effects been achieved? And how
c) Sustainability
• What are the main factors that facilitate or threaten the (financial and institutional) sustainability of the outcomes of the projects and activities?
• What are the main challenges in safeguarding the sustainability of the project?
• What are the main lessons learned from the project that could be documented? And to what extent could the project be scaled up or replicated in other regions?
• Is there a commitment of local institutions to guarantee the sustainability of the project?
• How has the work impacted the wider, non-material dimensions of well-being?

H. Expected Deliverables
The external evaluation consultant/team will be required to deliver the following key deliverables.
a) Inception report: The inception report will serve as an agreement between parties on how the evaluation will be conducted. Items to address:
• Appropriate validated draft data collection tools (e.g. methodological guidelines, group interview questions, Key Informant Interview questions guide)
• Raw data in any of the following statistical packages (STATA, SPSS, cSPro) and also transcribed qualitative scripts
• Understanding of the issues and questions raised in the ToR
• Data sources; how to assess the questions in the ToR
• Research methodology, including suggested sample and size
• Schedule of activities and traveling (timeline)
• Detailed budget
b) Facilitate a workshop for the presentation and validation of findings and recommendations to DKH and CPD.
c)(Draft and Final) Evaluation report of max. 25 pages (excluding annexes) to be structured as follows:
• Executive Summary
• Summary Evaluation report highlighting the cross-cutting key findings, lessons learned, and recommendations, including
• Evaluation purpose
• Evaluation methodology
• Main findings (presented in terms of achievements based on the process and impact indicators and their alignment with the project objectives & challenges encountered during the implementation of the various activities.)
• Stories of change and quotes from respondents
• Lessons Learned
• Conclusions and Recommendations
a. Regarding future steps/activities / follow up
b. Regarding improvements in project design
c. Regarding approaches and methodologies
Recommendations should be based on evidence and analytically sound. They should be targeted to the relevant audience, and they should be relevant and feasible)
I. Annexes
• Relevant maps and photographs of the evaluation areas where necessary (Photos should be Geo-tagged)
• Bibliography of consulted secondary sources
• Finalized data collection tools
• List of interviewees with accompanying informed consent forms
• PowerPoint presentation of preliminary findings to DKDH and CPD.

J. Timetable of evaluation
Indicative Timeline
The evaluation consultant /team will provide the deliverables according to the following indicative timetable:

Deliverables and Schedule Tentative Date Working days indication
Selection of the consultant 6-9 November 2023
Discussion with the Consultant (Virtual) on the Project and the TOR and contract signing 10 November 2023
Inception Report (including the evaluation methodology and detailed work plan) 20 November 2023
Data Collection and Field visits and. 21 November 2023
Submission of Draft Final evaluation report 30 November 2023
Final evaluation report submission 5th December 2023

K. Scope of Price Proposal and Schedule of Payments
The consultant shall be paid the consultancy fee upon completion of the following milestones.
• 40% after the adoption of the inception reports
• 60% after the approval of the final reports

L. Assignment Roles and Responsibilities
Consultant DKH/CPD
1. Develop an inception report, detailing the methodology, design, and instruments of the evaluation including data collection tools, stakeholders to be interviewed, and time frame for the evaluation.  Provide all required background materials to the consultant promptly.
 Read and provide comments on the inception report including the proposed research methodology, the information-gathering techniques, and the suggested target sites.
 Providing data/information for desk review.
2. In charge of the overall management responsibility of the review, including designing and carrying out the evaluation, drafting the final report, and debriefing the project team and key stakeholders.  Review and comment on deliverables
 Provide guidance where necessary.
 Support logistics for the evaluation team where necessary
 Facilitate contacts with key beneficiaries and line ministries.
 Arrange logistics and planning of the field research, support the evaluation team during fieldwork, and bring the research team to the beneficiaries.
3. Liaise with DKH/CPD staff throughout the process, providing weekly updates and seeking their input and advice where necessary.
4. Request approval in case of deviation from the budget, and for miscellaneous costs.
 Guide all phases of execution, and facilitation of the fieldwork, including interviews with local government, communities, etc.
 Review and comment on the draft reports submitted by the evaluator i.e, preliminary reports and the final report, providing feedback to draft data collection tools and reports.
 Review and provide feedback on the project evaluation report.
 Approving all deliverables and facilitating access to any documentation (or any person) deemed relevant to the evaluation process.
5. Final reporting and handing over of the final report  Final review, approval, and payment of the final installment of the consultant fees.

M. Required Expertise and Qualification
The consultant should have the following qualifications, skills, and experience:
 University degree in Education, Development Studies, Social Sciences, or equivalent qualification. Ideally, the persons will have experience in conducting Education evaluations and assessments. Education background, at least a Bachelor’s/Master’s degree in education or equivalent is desired
 Extensive experience in conducting surveys and knowledge of sampling methodologies
 Experience in community-based development approaches / participatory methods.
 Excellent communication and reporting skills
 Team Leader experience.
 Computer literate with skills in data analysis software SPSS.
 Good command of both written and spoken English.
 Familiarity with the Somalia context and education sector in particular
 Familiarity, ability, and willingness to travel within Somalia and under armed escort.
 Extensive experience in teacher training particularly in crisis contexts

N. Time frame
The evaluation is expected to start in November 2023 with an inception phase followed by intensive data collection (desk review, interviews, and surveys), analysis, and report writing. A virtual meeting for presentation and discussion of preliminary findings should be conducted on 26 November 2023, and the final revised evaluation report should be delivered by 1st week of December 2023.

O. Selection criteria
The selection for the suitable consultant will be based on scoring on awarding set marks for experience in undertaking similar assignments in education, especially in the complex humanitarian emergencies in Somalia, production of past consultancy assignments and references, and ability to interpret the consultancy assignment and add value to the methodology proposed and required outcomes among other parameters. The consultant shall be selected based on meeting the requirements provided for the assignments as indicated in the sections above.

P. Required Application Documents
The consultant shall be required to provide the following.
1. Technical and Financial Proposal (in USD) with clear breakdown including VAT in USD.
2. Updated CVs with 3 referees at least.
3. Registration and Tax compliance certificates from the Federal Government of Somalia