A. Project Background

Project context and rationale

Managed and run by NRC’s ICLA programme, the Protection and Return Monitoring Network (PRMN) is a coordinated system of humanitarian agencies providing a range of protection services designed to inform humanitarian planning, trigger appropriate protection responses, facilitate advocacies at different levels and enhance assessment capacity across Somalia. It serves as a platform for identifying and reporting displacements, in some cases returns, and protection incidents occurring as a consequence thereof. On an overall, PRMN seeks to achieve three central outcomes: Availability of information necessary to inform responsive and strategically targeted humanitarian response in Somalia, enhanced access to emergency protection assistance for populations victimized by serious protection incidents, and joint multi-sectoral assessment facilitated. Major service components include displacement and protection incident monitoring, provision of emergency protection assistance, referrals, post-return monitoring, service mapping, protection incident monitoring, ongoing assessments, as well as alert and reporting in real-time.

Thirty-eighty network organizations will maintain an active operational presence in 19 regions covering an aggregate 117 districts across Somalia: six (6) local partners in Somaliland, seven (7) in Puntland, nine (9) in Jubaland, and fifteen (15) across South Central Somalia. In total, 209 skilled and highly trained monitors are deployed across Somalia to facilitate monitoring and other project operations. Each partner organization is assigned a specific geographic coverage within a given region. The extent of coverage and ability partners to facilitate activities within a given area depends to a degree on the local security situation. There are also periods when certain regions will experience reduced reporting due to external events, changes in field partners or other reasons.

Data collection is accomplished through in-person interviews with members of the affected populations or key informants using a standardized questionnaire, usually at points of arrival, IDP settlements, transit centres and other strategic locations. For displacement incidents, data collection focuses on household-level information, while protection incidents involve the capturing of data unique to the persons of concern. Verified and approved data are uploaded into a web-based platform managed by NRC. Referral services and essential emergency protection support are available through the network to victims and survivors of severe protection incidents.

The PRMN methodology enables reporting on population movements and displacements together with analysis of trends over time and displacement from or to specific areas. The earlier forms of Network that existed in Somalia since 2006, Population Movement Tracking (PMT) and Protection Monitoring Network (PMN), were ultimately merged in 2010 to form the Protection and Return Monitoring Network (PRMN). It is, therefore, possible to perform trend analysis and comparisons over time can, but within certain limitations. Extensive geographic presence and the ability to identify and report displacement and protection incidents in real-time represent a unique added value of PRMN to the humanitarian response in Somalia – a context characterized by recurring emergency. Systems and procedures are in place that allow field monitors in the field to flag critical issues to NRC focal points, who verify and, together with UNHCR, may circulate a ‘flash report’ or ‘flash alert’ to the broader humanitarian community. The breadth of coverage of the network combined with the capture of origins, destinations and causes of movements means that the Network can provide meaningful insight into displacements covering a significant portion of the country.

PRMN, however, has some limitations. At the moment, the Network does not collect data intended to estimate IDP population at any given location. Therefore, while it may inform analysis and contribute in meaningful ways to the process of determining IDP figures, it is not a platform for estimating total or cumulative IDP populations in Somalia. At the same time, the network does not capture all population movements across Somalia at all times. Displacement reports and figures generated by the Network should, therefore, be considered illustrative indicators of potentially larger movements and their underlying causes. The network may not readily identify short-term displacement incidents where individuals or groups get displaced but quickly return. Albeit a single reason is recorded for each movement as the central driver, a combination of closely interrelated factors may trigger a movement.

A thorough technical review of operational modalities and service components is carried out annually as part of a standard quality assurance procedure. The exercise seeks to reinforce a balanced understanding of the factors underpinning displacement and to ensure that the Network remains relevant, responsive and appropriate. Where NRC and UNHCR consider that data is insufficient to provide results for a specific time-frame, data for a given location may be omitted from published reports (but may still be used in aggregated trends analysis). Throughout its evolution, PRMN has been shifting its focus in response to population movements and displacements in Somalia, from data collection to more detailed analyses to address prevailing humanitarian situations and inform durable solutions planning.

Project outcome:

NRC’s overall objective in Somalia is to promote and protect the fundamental rights of returnees, IDPs and vulnerable host communities and to facilitate voluntary return or reintegration as a durable solution, by focusing on the most recent and the most vulnerable returnees and IDPs.

Project outputs:

More specifically, NRC seeks to avail information necessary to inform responsive and strategically targeted humanitarian response in Somalia, increase access to emergency protection response, and strengthen protection assessment capacity in Somalia.

The programme has the following specific objectives:

o Monitor, report, triangulate, and verify data on displacements, population movements and avail information necessary to inform responsive and strategically targeted humanitarian response in Somalia**

o Expand emergency protection responses for populations of concern, in particular victims of serious protection violations.**

o Strengthen protection assessment capacity in Somalia through technical and material support**

o Build the capacity of local partners on protection and return monitoring, financial management, administration, literacy and computer skills as an effective way of delivering on projects.

o Support local partners with a Small Grant Fund for the implementation of the PRMN project

o Provide local partners with Emergency Grant Fund as a mechanism to support survivors of protection incidents and victims of protection and human rights violations

o Use the data generated from the PRMN system to undertake and inform protection and advocacy through dissemination of information to target groups and stakeholders

o Raise awareness on the socio-economic and humanitarian situation of returnees, IDPs and host populations among local and national administrative authorities, the humanitarian community, and other NRC projects, contributing to a better coordination of humanitarian and development interventions

B. Purpose and phase for the evaluation and intended use

The primary purpose of the evaluation is to provide an independent assessment of the effectiveness, impact, relevance and sustainability of the activities of the PRMN project since its inception in 2006. The review will facilitate the elaboration of an enhanced programme lay-out and advance practical recommendations replication in the region and potentially in other humanitarian contexts. The evaluation team should provide the Country Management Team with useful information, analysis and guidance that would enable the organization to engage in meaningful policymaking, effective planning, and an overall improved programming delivery. Having been in existence for more than a decade, NRC Somalia has decided to have the PRMN project externally evaluated with the view of, among other things, improving the programme design and documenting best practices, thereby providing a coherent framework for expansion within the East Africa and Yemen region. Moreover, access to information in a country riven by decades of war and an East African region with recurring humanitarian crisis is becoming increasingly challenging. Given the current political situations in Somalia, neighbouring Ethiopia and Yemen, and other countries such as South Sudan, Sudan, Burundi and DR Congo, and pattern of returns, effective humanitarian response will require enhanced access to information and protection analysis to inform programming, including effective targeting.

Intended use of results

On an overall, the evaluation seeks to document PRMN best practices and advance framework-wide recommendation(s) for improvement. Once systematically documented, the findings will be used by NRC and UNHCR to reorient, refine and/or adjust, where necessary, PRMN’s focus and methodology in Somalia, and to facilitate a potential replication of the framework in other parts of the region as a best practice of displacement monitoring information system.

C. Scope of work and methods

The evaluation should cover the overall assistance provided through ICLA from January 2006 to December 2019 as part of the PRMN project. As parallel objective, the evaluation seeks to implement a comprehensive review of PRMN’s approach and services in Somalia, its impact to date, as well as the extent the current methodology will need to be improved to ensure higher impact and effectiveness and continued relevance in years to come.

The methodology will include:

1. Desk studies as general background, the evaluation team should study relevant internal strategic documents, including NRC Somalia’s country strategy, appropriate action plans, project applications, correspondence, agreements and reports.

2. The evaluation was expected to include a field mission intended to facilitate interviews with PRMN monitors, target groups, beneficiaries of the emergency protection assistance component of the project, local authorities and members of host communities assisting with triangulation of information, national and international staff of the ICLA programme, representatives of federal and local governments, selected heads of humanitarian organisations, donor representatives, UNHCR and other UN agencies. However, it is unlikely that the COVID-19 crisis will be over during the timeframe specified for this assignment. Therefore, NRC will organize the logistics necessary to facilitate remote data collection. Interviews, meetings and other group sessions will be held via online teleconferencing platforms. The evaluation team will still be able to assess PRMN activities, including but not limited to:

o Information collected and stored in PRMN online system,

o Types of reports and documents generated through the PRMN system

o Mechanisms and processes used for data collection

o Advocacy initiatives and humanitarian responses informed by PRMN reports.

In so doing, the evaluation will assess as to whether PRMN information services and protection response were relevant and adequate, whether PRMN improved overall humanitarian planning and response in Somalia, and importantly whether the system adequately adapted to the changing humanitarian context.

Evaluation principles

The following ethical rules/considerations will guide the evaluation process.

1. Openness – of information given, to the highest possible degree to all involved parties
2. Publicity or public access – to the results when there is no special consideration against this
3. Broad participation – interested parties should be involved when relevant/possible
4. Reliability and independence – the evaluation should be conducted so that findings and conclusions are correct and trustworthy.

D. Issues to be covered

The evaluation team will assess the performance, relevance and impact of the PRMN project in Somalia by applying the criteria described below. These criteria are also clearly defined in NRC’s Evaluation Policy. The questions under each criterion are not to be construed as an exhaustive list. Instead, they are intended primarily to guide the evaluation team in focusing on the issues that are of significant interest for NRC.

1. Relevance/appropriateness

At the inception of PRMN, Somalia was immersed in a humanitarian crisis characterized by a fragile peace, recurrent hostilities, internal mass displacement, as well as spontaneous and forced return of Somalis from other countries. While the current humanitarian context remains largely unchanged, there is relative stability, and Somalia is transitioning to durable solutions programming while dynamic such as eviction resulting in forced secondary displacements, has become prevalent. Amid such intertwining dynamics, the relevance of the evaluation is anchored on the following:

o Was a thorough assessment focusing on contextual relevance undertaken before the design of the project? If yes, to what extent is PRMN aligned to the local environment, and humanitarian and protection needs identified?

o Given the extreme poverty in which most Somalis live, including NRC target population, did potential beneficiaries and host communities participate meaningfully in defining how the PRMN project could respond to their needs? Was such participation and involvement necessary for an information-type project such as PRMN?

o To what extent does PRMN contribute in providing appropriate responses to the needs of NRC’s target populations (returnees and IDPs) on the one hand, and the wider humanitarian community, on the other?

o Are the goal and objectives of PRMN in line with NRC ICLA Policy and/or contribute to it?

o Are the objectives of the project in line with and contribute to NRC’s overall programme objectives?

o Has NRC the required capacity in terms of staffing, local knowledge and experience in the country to effectively implement PRMN?

o What are other technically sound alternatives to PRMN’s current design in terms of information management in displacement context? Is PRMN the best alternative in the Somali context. Why or why not?

o Are there functioning mechanisms in place to monitor whether the project consistently adapted to context changes and the information needs of the humanitarian community in Somalia?

o What aspects of PRMN can be categorized as best practice on a good displacement monitoring information system? How and what other elements/components of PRMN should be improved, if any?

o What aspects of PRMN can be replicated in different humanitarian contexts, and to what extent?

2. Effectiveness

The outcome envisaged of the PRMN project in Somalia is to inform humanitarian planning, trigger appropriate protection responses by other service providers, provide direct emergency protection assistance, facilitate advocacies at different levels, and enhance assessment capacity across Somalia. This overarching goal is achieved by adhering to the following foundational questions.

o Are objectives and activities sufficiently and clearly defined? Are they relevant to the context and the envisaged outcome of the project?

o Has the project set criteria for selecting beneficiaries as per its objectives? If yes, have they been applied consistently?

o Is there an internal monitoring mechanism consisting of objectively verifiable indicators against which performance, quality and impact are assessed?

o To what extent has the PRMN project achieved its original and subsequent adapted objectives?

3. Efficiency

o To what extent the project has efficiently utilized its resources and time?

o Is the direct implementation model where NRC manages partners the best alternative? Would other modalities, i.e. the use of independent implementing partners, have improved the balance between inputs and outputs?

3. Coordination

• To what extent PRMN coordinates with other data and information management actors in Somalia, both at the federal and sub-national level?
• Has coordination with NRC Kenya and NRC Ethiopia been relevant to the objective of enhancing cross-border cooperation, information and assistance to Somalia refugees in neighbouring countries, and returnees and IDPs in Somalia?

4. Impact

o Has the assistance provided by PRMN helped humanitarian actors to make well-informed decisions in terms of planning and response?

o Has PRMN and its components contributed to protection and durable solutions?

o What consequences has PRMN had on the population of concerned – direct and indirect, intended and unintended, and positive and negative?

o Are there quantitative and qualitative indicators to measure the impact of PRMN activities? Are there monitoring and analysis mechanisms in place?

5. Sustainability

o Has the project identified exit strategies?

o To what extent are those in line with NRC policies?

6. Protection and durable solutions

o To what extent does the PRMN project respond to existing protection issues for NRC beneficiaries (IDPs, returnees, deportees, refugees)?

o To what extent have advocacy efforts, initiated by or as a result of PRMN data, achieved positive and timely results?

E. Evaluation team and steering committee

An independent evaluator, or a team of professional evaluator with legal, protection and human rights expertise on East Africa and Yemen region, will undertake the evaluation and be responsible for writing the report. At a minimum, the evaluation team should have evaluation experience related to legal aid, protection, and/or human rights projects and/or experience with situations of forced displacement. The consultant should have professional knowledge about the conflict and culture in Somalia, and be gender-sensitive. Fluency in English is a strict requirement for the individual or team that will undertake the evaluation. The report will contain the difference of opinion among members of the evaluation team in terms of conclusions and recommendations.

A Steering Committee will be established, with the following members:

1. Barnabas Asora – Chairperson, Head of Programme – Somalia
2. Joseph Jackson – ICLA Specialist, Evaluation Manager
3. Evelyn Aero – Regional Technical Adviser
4. Nicolas Cozza – Regional M&E Adviser
5. Mohamed Hassan – Somalia M&E Manager

The primary function of the Steering Committee will be to select the external evaluators, review preliminary findings and recommendations and establishing a dissemination and utilization strategy. The main function of the Evaluation Manager will encompass the preparation and approval the Terms of Reference (in close collaboration with the stakeholder and members of the steering committee), administration and overall coordination, including monitoring progress.

F. Timeframe and budget considerations

The whole process of the evaluations will have a time frame of six weeks (46 working days) starting 7^th July 2020 and ending 22 August 2020. The evaluation team will begin its work precisely at the contract start date, working remotely. The evaluation team should contact the Evaluation Manager at NRC immediately if serious problems or delays are encountered. Approval for any significant changes to the evaluation timetable will be referred to the Steering Committee.

G. Reporting

At the end of the desk review and data collection, the evaluation team will hold a virtual workshop with the ICLA project team and other relevant staff identified by the Steering Committee to discuss the preliminary findings of the evaluation exercise. A draft report should be submitted not later than 08 August 2020.

The completion date for the Final: The consultant will disseminate the final evaluation report on 18 August 2020, having addressed all comments as appropriate. The size of the report shall be approximately 40 pages, appendices not included, clearly written in English using Arial 11 point.

The evaluation report should consist of:

· Executive summary and recommendations not more than six pages.

· Main text to include an index, emergency context, NRC mandate, evaluation methodology,

· Commentary and analysis addressing evaluation purpose and outputs to include a section dedicated to the issue of particular lessons-learning focus, conclusions (not more than 35 pages)

· Appendices, to include evaluation terms of reference, maps, sample framework, and bibliography)

· The consultant shall lodge with the Evaluation Manager before the end of the contract, all material collected while undertaking the evaluation.

H. Follow up and Management response

For the follow up of the evaluation, the ICLA Specialist in Somalia is the main responsible, with the Regional ICLA Adviser for East Africa and Yemen as the focal point at NRC Regional Office. Implementation of PRMN in 2020 and beyond will take into account the conclusions emerging from the workshop with the evaluation team. A management response, responding to the recommendations, including an action plan should be prepared by the ICLA Specialist no later than two months after receiving the final report. It is the responsibility of the Country Director, Somalia to ensure that the realizations of these plans are monitored and documented.