TERMS OF REFERENCE – FINAL EVALUATION – ENHANCING INTEGRATION OF DISPLACEMENT AFFECTED COMMUNITIES IN SOUTH WEST STATE OF SOMALIA (EIDACS “A” and EIDACS “B”)

Funded by the European Union

Contents

1. INTRODUCTION.. 4

2. DESCRIPTION OF THE PROGRAMME. 5

3. EVALUATION PURPOSE. 8

4. TARGET AUDIENCE. 9

5. SCOPE OF THE EVALUATION.. 9

6. METHODOLOGY. 10

7. WORKPLAN AND TIME SCHEDULE. 12

8. EXPECTED DELIVERIES. 14

9. RESPONSIBILITY AND AUTHORITY. 14

10. LIMITATIONS. 16

11. QUALIFICATIONS OF THE CONSULTANT. 16

12. APLICATION PROCESS AND REQUIREMENTS. 17

List of Acronyms

APT

Area Programme Team

CMU

Coordination Management Unit

DAC(s)

Displacement Affected Communities

DAC

Development Assistance Committee(OECD)

Danida

Denmark’s Development Cooperation

DRC

Danish Refugee Council

DS

Durable Solutions

DSP

Durable Solutions Platform

EIDACS

Enhancing Integration of Displacement Affected Communities in Somalia

EUTF

European Union Emergency Trust Fund

FGD

Focus Group Discussion

GREDO

Gargaar Relief and Development Organization

HLP

Housing, Land and Property

HoA

Horn of Africa

IASC

Inter-Agency Standing Committee

IDP

Internally Displaced People

KII

Key Informant Interview

MoPIC

Ministry of Planning and International Cooperation

M&E

Monitoring and Evaluation

NGO

Non-Governmental Organization

NRC

Norwegian Refugee Council

OECD

Organization for Economic Co-operation and Development

ReDSS

Regional Durable Solutions Secretariat

UN-RCO

United Nations Resident Coordination Office

SDSC

Somaliland Durable Solutions Consortium

SHACDO

Shabelle Community Development Organization

SPL

Somali Peace Line

SWS

South West State

ToR

Terms of Reference

VSLA

Village Savings Association Loan

WASH

Water, Sanitation and Hygiene

1. INTRODUCTION

Concern Worldwide was awarded two contracts totalling €5 million by the European Union (EUTF) as Consortium Lead: Enhancing Integration of Displacement Affected Communities in Somalia, “EIDACS” (also known as EIDACS A) in South-West Somalia – Baidoa and Afgoye [T05-EUTF-HoA-SO-03-4.4(T05.241)]; and Enhancing Integration of Displacement Affected Communities in Somalia, Baidoa, “EIDACS-B” [T05-EUTF-HoA-SO-03-4.6(T05.362)].

These programmes are implemented through a Consortium formed by Concern Worldwide (as Lead agency), Gargaar Relief and Development Organization (GREDO), Norwegian Refugee Council (NRC), Danish Refugee Council (DRC), and the Regional Secretariat for Durable Solutions (ReDSS).

This is a Durable Solution Consortium and the implementation areas are specific IDP sites in the urban locations of Baidoa and Afgoye, with the objective to create a conducive environment to reach durable solutions for displacement-affected communities. [1]

These Terms of Reference (ToR) have been prepared to undertake the Final Evaluation of EIDACS A and EIDACS B. It is not a donor requirement, but the interest of Concern Worldwide and the Consortium to go through a deep reflection process after 3 years of a Durable Solutions pilot experience in a very new and innovative sector.

This evaluation will happen at the same time others are on-going: EUTF Mid-Term Evaluation, REDDS Lessons Learnt Documentation Process, Somalia Durable Solutions Consortium Project (SDSC) End line Evaluation, Danida Durable Solutions Programme(DSP) Mid-Term Evaluation, EU/Re-Integ Third Party Monitoring by Axiom and Altai, amongst others. It will therefore not be a stand-alone process, and will complement other existing and ongoing monitoring processes, evaluations and researches in the Durable Solutions sector in Somalia.

This will be an External Evaluation, conducted by an External Consultant/Consultancy, recruited through a tender process. The evaluation is expected to be implemented between March and April 2020.

The External Consultant/Consultancy will share the findings, conclusions and recommendations, through the distribution of executive summaries and full reports, during stakeholders events organized for this purpose. The evaluation results might be further disseminated through a documentary (or short documentaries). The documentary is not included in this ToR, but is part of a separate document and process, with its specific ToR.

2. DESCRIPTION OF THE PROGRAMME

EIDACS A and EIDACS B are implemented through a Durable Solution (DS) Consortium, with the following implementing partners: Concern Worldwide (Lead agency), Gargaar Relief and Development Organization (GREDO), Norwegian Refugee Council (NRC), Danish Refugee Council (DRC), and the Regional Secretariat for Durable Solutions (ReDSS) as Learning Partner. Moreover, under EIDACS A, the Consortium has contracted the services of Somali Peace Line (SPL) -Tubta Nabadda Soomaaliyeed, to develop the peace building component; and activities in Afgoye are implemented thorough Shabelle Community Development Organization (SHACDO).

EIDACS A is implemented in two incubator locations[2] in Baidoa (named Hannano and ADC) and one in Afgoye (Dollawe). EIDACS B is only implemented in the incubator locations of Baidoa.

EIDACS A contract start and end dates: 1 March 2017- 29 February 2020. Currently, a Non-Cost Extension (NCE) for three months has been requested. If approved the end date will be postponed to May 2020. EIDACS B contract was approved a few months later than EIDACS A, to address the needs of a new influx of internally displaced people arriving in Baidoa. EIDACS B started on 1 August 2017 and will finish on 31 July 2020. However, since EIDACS B had a fast implementation and was aligned with EIDACS B from the beginning of the intervention, the activities of both EIDACS A and B will finalized at the same time, by 29 February 2020. With the NCE, few activities implemented by Concern will remain until May 2020. Therefore, this evaluation will cover both programmes.

Beneficiaries

The targeted beneficiaries of the project include IDPs, members of the host communities, returnees and refugees. The original number of beneficiaries in the proposal was calculated based on population estimates in the various districts/ regions, including various secondary sources and EIDACS rapid assessment on the situation of IDPs in target districts conducted in May 2016: 25,804 (14,192 IDPs / 10,322 Host Communities / 1,290 Returnees).

By August 2019, EIDACS A had already doubled this number, reporting 51,608 beneficiaries, out of which 33,742 are IDPS and 16,544 are members of the host communities. Data collection has only included the number of displaced and non-displaced, without distinguishing between IDPS, refugees or returnees. Out of the total number of beneficiaries reached by August 2019, 63.74% of them were women.

EIDACS B also estimated to target 25,804 beneficiaries and by July 2019, the Consortium had reported 45,751 beneficiaries (31,711 IDPSs and 14,040 members of the host communities). Out of the total number of beneficiaries reached by July 2019, 68% of them were women.

The number of beneficiaries overlap between outputs and between EIDACS A and EIDACS B.

Outcomes and Implementing Partners

EIDACS A is a multisector programme implemented in Baidoa and Afgoye, as follows:

Outcome

Outcome Title

Baidoa

Lead Implementing Agency

Afgoye

Lead Implementing Agency

1

Displacement-affected communities (DACs) are able to influence decisions, policies and agreements that affect them collectively as well as where to live and how they are governed

Op 1.1: Inter-community agreements are made on DAC issues and durable solutions plans

Op 1.2: Restoration of Housing Land and Property

DRC

NRC

Concern Worldwide

2

DACs have the same access and use of basic services/material safety as other non-displacement affected communities

Op 2.1: DACs have access to primary education of adequate condition and quality

Op 2.2: DACs have access to basic health services

Op 2.3: DACs have access to basic WASH Services

Concern Worldwide

GREDO

DRC

Concern Worldwide

3

DAC have the same access to adequate livelihoods through generating income and assets, gainful employment, and managing financial risk as other non-displacement affected communities.

Op 3: Target groups have improved productive livelihoods to a targeted standard

DRC

Concern Worldwide

4

Learning on best practices and lessons on Durable Solutions disseminated by EIDACS programming are utilized by actors and stakeholders working in Somalia.

Concern Worldwide

ReDSS

Concern Worldwide

ReDSS

Every Consortium partner is implementing activities in Baidoa while Concern is the only agency implementing in Afgoye.

EIDACS A is also implementing two group of activities, which were not detailed in the proposal nor included in the logic framework, and were developed during the implementation:

1) Peace Building activities by Somali Peace Line (SPL); and

2) Joint Monitoring visits with the Ministry of Planning, Investment and Cooperation (MOPIC), South West State of Somalia (SWS).

These interventions are also under the scope of this evaluation.

EIDACS A is the “umbrella” programme under which EIDACS B was designed, for an emergency provision of basic services.

EIDACS B is also a multisector programme, only implemented in Baidoa, as follows:

Outcome

Outcome Title

Baidoa

Lead Implementing Agency

1

Displacement-affected communities (DACs) are able to influence decisions, policies and agreements that affect them collectively as well as where they live.

Op 1: Restoration of Housing Land Property (HLP).

*Op 1.2: Drought affected populations have enhanced access to specialized services, are able to identify, claim, and exercise their HLP rights, and address post-eviction shocks**.*

NRC

2

Outcome 2: DACs have the same access and use of basic services/material safety as other non-displacement affected communities

Op 2.1 DACs have access to primary education of adequate condition and quality.

Op 2.2 DACs have access to basic Health and WASH.

Op 2.3 DACs have access to basic WASH Services

Concern Worldwide

GREDO

DRC

From now on, “EIDACS” will be used to refer to both EIDACS A and EIDACS B.

3. EVALUATION PURPOSE

The purpose of this evaluation is to:

· Measure the effectiveness of the programme through assessment of progress towards expected outcomes.

· Document and capitalize[3] programme and consortium best practices and lessons learnt.

· Make recommendations on key measures of Durable Solutions that could be included as indicators in future programmes.

· Make recommendations to inform future programming, donors and other DS consortiums and programmes.

· Identify potential future gaps after the end of the contract and contribute with recommendations to the exit strategy.

· Show accountability, give visibility, and contribute to the DS and Humanitarian debate in Somalia.

4. TARGET AUDIENCE

The final evaluation will provide useful information to the main stakeholders involved in the programme. The “users” of this evaluation will be:

· DAC Forums, direct and indirect beneficiaries

· Consortium Partners: Concern, NRC, GREDO, DRC, ReDDS

· Other partners: Somali Peace Line and SHACDO

· Consortium Donor: EUTF

· Other donors: Danida, DFID

· UN agencies working in DS: UN RCO DS Unit, UNHCR, UN-Habitat.

· Different Government levels of Somalia: DS Secretariat FGS, MOPIC SWS, Baidoa District Commission and Municipality

· Other DS Consortia / Programmes: Re-Integ DS Consortia, Danwadaag, DSP

· Concern Worldwide as the Lead Agency

5. SCOPE OF THE EVALUATION

The scope of the evaluation includes EIDACS A, implemented between 1 March 2017 and 29 February 2020[4] and EIDACS B, implemented between 1 August 2017 and 29 February 2020, in two incubator locations in Baidoa (ADC and Hannano: 18 sites) and one incubator location in Afgoye (Dollawe).

Together with the objectives, outcomes and indicators of the logical framework, this evaluation will measure the peace-building component implemented by SPL and the Joint Monitoring activities led by MOPIC SWS.

6. METHODOLOGY

This evaluation will include 1) an analysis of the processes, looking at how has the programme been implemented; 2) an analysis of the programme against DAC Criteria and IASC/ReDDS DS Criteria; and 3) a cost-effective analysis and cost-benefit analysis of the intervention.

The External Consultant will complete the following Evaluation matrix, with specific research questions, the tools to be used and the indicators, in consultation with the stakeholders and will include it in the Inception report.

A draft should be shared as part of the Technical Proposal of the Application, included the added value beyond this ToR.

DAC CRITERIA

KEY EVALUATION QUESTIONS

Relevance

• To what extent are the objectives of the programme still valid?
• Are the activities and outputs of the programme consistent with the overall goal and the attainment of its objectives?
• Are the activities and outputs of the programme consistent with the intended impacts and effects?

Effectiveness

• To what extent were the objectives achieved?
• What were the major factors influencing the achievement or non-achievement of the objectives?

Impact

• What has happened as a result of the programme?
• What real difference has the activity made to the beneficiaries?
• How many people have been affected?

Efficiency

• Were activities cost-efficient?
• Were objectives achieved on time?
• Was the programme implemented in the most efficient way compared to alternatives?

Sustainability

• What are the major factors, which can influence the achievement or non-achievement of sustainability of the programme?​

The External Consultant will complete the Evaluation Matrix and will develop the approach and methodology accordingly. This will be included in the inception report and will need to be validated by the Evaluation Team[5] before moving to the next phase of the evaluation.

The methodological approach for data collection will combine a mix of both quantitative and qualitative data.

Quantitative data will be drawn from project documents and reports, including external reports (Altai, ReDSS, etc.), and incorporated into the analysis. Moreover, data will be drawn for the Surveys conducted during the Programme implementation: Concern M&E Team conducted three Annual Surveys using the Digital Data Gather System (August 2017, March 2018 and April 2019). These surveys included questions to measure key indicators outlined in the M&E plan for the programme (Concern M&E Team will conduct an end-line Survey by the end of the February-beginning of March). The external consultant will analyse the four surveys.

The indicators measured under these Surveys go beyond the Logical Frame indicators and provide useful information on aspects like social cohesion, gender attitude, and sense of feeling (un) safe or participation in public affairs, amongst others. Automatic calculations of key indicator values are available in the DDG summary reports available online. However, no comparative analysis has been done on the evolution of indicator values over time, nor any analysis to determine whether any changes are statistically significant. The analysis by the Consultant will need to determine if the results are statistically significant and if attribution to the programme can be inferred.

Qualitative data will be collected through observation, FGDs, KII and stakeholder meetings. Considering the variety of views and interests of stakeholders of the evaluation, the stakeholders’ perspectives will be triangulated for many of the evaluation questions in order to bolster the credibility and validity of the results.

The Technical Proposal for this Application should include added value beyond this ToR, including original ideas on the methodology that will be used.

Meetings, interviews and FGDs must be conducted by the External Consultant in Nairobi, Mogadishu and Baidoa (all-inclusive).

The Consultant will also analyse the data included in the surveys under a Durable Solution perspective, against ReDSS/ IASC DS Framework (See Annex List-of-ReDSS-framework-indicators.pdf)

The findings, conclusions and recommendations will be shared by the External Consultant through the distribution of executive summaries, full reports and a power-point presentation, during a stakeholder meeting organized by the External Consultant and the Evaluation Team for this purpose.

Moreover, and out of this ToR, the evaluation results will be further disseminated through a documentary (or short documentaries). The documentary is not included in this ToR, but is part of a separate document and process, with its specific ToRs.

Main Actors that will be involved in the Evaluation (key informants)[6]

• Consortium Technical Work Group/ Area Programme Team
• ReDSS as Consortium Learning Partner
• Consortium Steering Group / CDs or delegated staff
• Governmental actors: MOPIC SWS, Line Ministries, Baidoa District Commission and Baidoa Municipality
• UN actors: DS Coordinator/ Integrated Office of the UN Resident Coordinator for Somalia, UNHCR, UN-Habitat
• Other DS Programme: Danwadaag and DSP
• Donor/s: EUTF, DFID, Danida

7. WORKPLAN AND TIME SCHEDULE

The evaluation will be conducted in 37 working days, including the inception phase and desk review, field data collection, interviews in Nairobi, Mogadishu and Baidoa, draft and final report and final presentation. The assignment is expected to commence on March, with the final evaluation report expected before the end of 30 April 2020.

TASKS

RESPONSIBLE

TIMELINE[7]

Final ToR and launching tender

Validated by Concern Worldwide

Deadline 30 January 2020

Consultancy recruitment process

Concern Worldwide

Logistics/Procurement (international tender)

1-29 February 2020

Signing of Contract and beginning of Consultancy

Concern Worldwide

Logistics/Procurement (international tender)

1 – 15 March 2020

Inception phase, desk review and submission of the Inception Report

External Consultant/Consultancy

7 days since signature of Contract

Validation of the Inception Report (while logistics arrangements are ready for field visits)

Evaluation Team

3 days

Field work, quantitative and qualitative data collection

External Consultant/Consultancy

10 days

Draft report: share and request feedback; inputs are shared. Findings are contrasted and validated

External Consultant/Consultancy with the Evaluation Team

10 days

Feedback incorporated into the Final report.

Dissemination:

Final Report is circulated and a power point is prepared and presented by the External Consultant

External Consultant/Consultancy and Evaluation Team

Stakeholders to participate in the presentation

7 days

8. EXPECTED DELIVERIES

The External Consultant is expect to deliver the following:

• Inception report: including the Evaluation Matrix and the methodology
• Final Report, with the following contents:

· Title and cover page

· Acknowledges (if any)

· Table of contents

· List of acronyms

· Executive Summary

· Introduction and Background

· Purpose and scope. Limitations

· Methodology

· Findings and Conclusions

· Best practices and Lessons Learnt

· Compilation of the four Annual Surveys conducted and an analysis of the data, including justification of changes in trends

· IASC DS Framework completed

· Recommendations on key measures of DS that could be included as indicators in future programming

· Strategic and programmatic recommendations to inform future programming, donors and other DS consortia and programmes

· Identification of critical gaps after the end of the contract and recommendations for the Exit Strategy and transition to other DS programmes

· Annexes

• Power point Presentation for Dissemination of the Evaluation Final Report

9. RESPONSIBILITY AND AUTHORITY

Concern Worldwide Somalia will establish an Evaluation Team to validate different stages of the evaluation process and support the necessary logistics in the field. EIDACS Consortium Coordinator will be the focal point for the External Consultant and the responsible for the overall coordination of all the evaluation tasks with the Consultant. In addition, the following members of Concern Worldwide Somalia will provide all the necessary technical and operational support required throughout the evaluation process: Program Director, M&E Manager, Durable Solutions Program Coordinator, Durable Solutions Program Manager in Baidoa, Desk Officer for Somalia in HQ and Concern M&E focal point in Dublin. Moreover, the partners will appoint one focal point to support during different phases of the evaluation.

Concern Worldwide Somalia will be responsible for:

• Recruit the External Consultant and finalize the Consultancy agreement
• Guide and coordinate the External Consultant throughout all the phases of the evaluation
• Share all necessary documents with the Consultant to prepare the Inception Report and throughout all the phases of the evaluation
• Provide inputs to the External Consultant and validate the Inception report
• Ensure that feedback from EIDACS Consortium partners is circulated and shared with External the Consultant
• Inform the Evaluation Audience and ensure their involvement in the evaluation; help in setting specific dates for the evaluation field schedule
• Book and cover travel expenses to Mogadishu, Baidoa and Afgoye
• Vehicle and security hire in Mogadishu, Baidoa and Afgoye
• Food and accommodation for the Consultancy Team in Mogadishu, Baidoa and Afgoye, at Concern Worldwide Guesthouse
• Working space for the Consultant in Mogadishu and Baidoa
• Follow-up the data collection process, ensuring quality control

The Consultant/Consultancy Team will be responsible for:

• Keep close communication, coordination and reporting with EIDACS Consortium Coordination, as Evaluation Team Leader
• Develop the methodology for this evaluation
• Provide the expected deliveries included in these Terms of Reference (in English Language)
• Develop the field work schedule in consultation with the Evaluation Team
• Comply with the agreed Work Plan and inform of any modifications
• Recruitment and payment of enumerators in Baidoa and Afgoye
• Conduct training for enumerators during the field visits phase
• Stationery for data collection
• Supervise the data collection process, provide advice and ensure the quality of the data
• Conduct interviews (KII) and FGDs with the main stakeholders
• Analyse and share the data collected and contrast findings with stakeholders
• Provide regular field briefing to the Evaluation Team on the progress and any challenges from the field.
• Book and cover flight ticket from country of origin (if different from Kenya or Somalia)
• Follow Concern Worldwide staff expenses procedures
• Obtain and cover insurance for the full period of the Consultancy
• Follow Concern Worldwide Security Rules and Procedures

10. LIMITATIONS

• Security may limit accessing areas, target beneficiaries and information, especially in Afgoye.
• Staff turnover might led to information gaps

11. QUALIFICATIONS OF THE CONSULTANT

We are looking for a Consultant with the following skills and qualifications:

• The Team Leader must possess at least a Master’s Degree or PH in any of the following fields: International Development, Social Sciences, Statistics, Community Development, Development Studies, Local Government or any related field
• Demonstrated knowledge in the Durable Solutions Initiative in Somalia
• Strong and documented experience in conducting participatory qualitative assessments related to community engagement; housing land and property; education, health and nutrition and water and sanitation; and livelihoods programming.
• Demonstrated experience in leading at least three similar project evaluations
• A solid understanding of remote management and learning
• Demonstrated experience in leading teams, training local staff in quantitative and qualitative data collection tools including entry template
• Demonstrated experience in designing survey methodology, data collection tools, processing and analysis of data
• Ability to interact with host government, partners as requested by Concern Worldwide Somalia
• Strong organizational, analytical and reporting skills, presentation skills, attention to detail, ability to meet deadlines, and proficiency in SPSS or other statistical packages, Microsoft Office and qualitative data analysis software/tools.
• Previous experience in Somalia or a fragile country with tight security context will be preferred.
• Excellent verbal and written communication in English required.

[1] The original proposal also included the location of Merca, which finally was not included in the intervention due to challenges in access.

[2] Incubators are clusters of IDPs at risk, identified through a community mapping exercise at the beginning of the project

[3] Capitalize, understood as gaining advantage to expand the value of the lessons; this capitalization may include concepts of innovation and human center design

[4] Assuming the NCE request is approved, EIDACS A will be implemented until May 2020. The timeframe to be evaluated will be agreed between the Consultant and Concern Worldwide during the inception phase.

[5] See composition of the Evaluation Team under section 10. Responsibility and Authority of this Terms of Reference

[6] Their roles will be further developed during the inception phase; some are already included under Section 7: Work Plan of this ToR.

[7] These dates are tentative and subject to the date of signature of the contract and availability of the External Consultant/Consultancy and the Evaluation Team. The Consultancy days will not necessarily be consecutive.