Terms of Reference

Project title: Public Private Development Partnership for Renewable Energy Skills Training and Women’s Economic Empowerment in Somalia

Project code: SOM1702MSWE

Donor: SIDA

Project duration: 4 years: 6 months inception, 3.5years implementation phase

Evaluability review schedule and location: April-May 2020 / home-based

Geographical coverage: Mogadishu and Garowe.

1. Introduction

In accordance with International Labour Organization (ILO) policy governing technical cooperation projects, an evaluability assessment is required for the Public Private Development Partnership for Renewable Energy Skills Training and Women’s Economic Empowerment project in Somalia. To support results-based management of ILO projects and programmes, it is critical that their design meet the minimum standards for monitoring and evaluation (M&E). This can be determined by assessing a set of design-specific aspects prior to implementation, which are defined as “evaluability.” This adheres to the OECD/DAC definition: “the extent to which an activity or a program can be evaluated in a reliable and credible fashion”[1]

The assessment will be carried out using the ILO’s evaluability instrument to score the Public Private Development Partnership for Renewable Energy Skills Training and Women’s Economic Empowerment project in Somalia on a set of M&E criteria and provide recommendations for improvement. It will review the programme’s M&E strategy in 7 areas:

1. Clarity of objectives/outcomes
2. Quality of Indicators
3. Establishment of baselines
4. Inclusion of time-bound milestones
5. Assessment of risks and assumptions
6. Comprehensiveness of M&E strategy
7. Validity of methodology for gender analysis
8. Contribution to SDGs

The assessment will be conducted in compliance with the principles and standards for project evaluation set forth in the ILO Policy guidelines for results-based evaluation.

2. Project Background

2.1 Origin and content of the Project

The International Labour Organization (ILO) signed two-component project targeting: women’s economic empowerment, specifically for growth-oriented women entrepreneurs in the MSME sector; and skills development to support an expanded electrical system with a greater proportion of renewable energy sourcing. Developing skills in the electrical sector and supporting women’s economic empowerment addresses two core limiting conditions for Somalia, but there are also opportunities to build employment opportunities and greater economic empowerment for women as part of the expansion of the electrical system.

Given the scale, complexity and overall budget of the ILO Public Private Development Partnership for Renewable Energy Skills Training and Women’s Economic Empowerment project in Somalia, it is important to check that the programme is robust and that it will be possible to measure and assess the results achieved following implementation. An independent evaluability assessment of the programme will be a useful programming and management tool as it will provide (i) an appreciation of the relevance, coherence and strength of the programme’s design; (ii) enable early adjustment and corrections to the design of the programme, as needed, (iii) strengthen the application of tools to track programme performance and demonstrate results and (iv) provide an indication of the readiness of programmes to be evaluated. This constitutes the overall goal of the evaluability assessment.

Conducting an evaluability assessment early on will validate the programme’s continuing coherence and check if the means to measure progress and demonstrate results are in place. The evaluability assessment helps to manage risks by clarifying gaps in overall programme logic, or pointing to a lack of data. The participatory approach taken for the Evaluability Assessment will foster a shared understanding among key stakeholders of the targeted results and the programme logic.

*2.2 Project results framework

The pilot phase provided a solid ground for the formulation of the project’s main phase results framework. The project comprises of three overall outcome areas structured around 5 output areas. During 2019, the project delivered on the below referenced activities:

2.2.1 Project stakeholder meetings

· In 2019, three Technical Working Group (TWG) were held These technical working groups guide the project implementation and act as an advisory board the project Steering Committee

· A Project Steering Committee (PSC) meeting was conducted in 10^th December 2019. It was attended by all relevant line miniseries; Ministry of Women and Human Rights development, Ministry of Labour and Social Affairs, Sweden and all other key stakeholders.

· Also, project continues to ensure visibility for SIDA and all other implementing partners, developed a communication strategy and conducted partner selection and stakeholder synthetization.

2.2.2 WEE component

Outcome 1: Women-owned SMEs are more viable and have effective institutional and peer support

Output 1.1: Financial and Non-Financial BDS providers and CSOs have the required capacities to deliver a wide range of gender-sensitive business development services to growth- oriented women entrepreneurs

Output weighting: 50%

· 100 BDPs and CSOs selected in Mogadishu, Garowe

· A gender self-assessment in one BDSP organization, Shuraako/OEF in order for them to facilitate the process with 9 other BDSPs/CSOs. A selection criteria for the 9 BDSPs/CSOs was developed.

· Start and Improve Your Business (SIYB) Training of Trainers (ToT) took place in Mogadishu in December 2019, with participants coming from Mogadishu, Garowe, Hargeisa, Baidoa and Bossaso. 25 trainers in total of which 8 are women have been trained. Out of the 25 participants, eleven (11) trainers were selected to go through to certification stage. The certification process is scheduled to take place between March 29^th to 9^th April 2020.

· The training on Improve Your Exhibiting Skills (IYES) was also conducted on December 15, 2019. 25 of which 5 are women, coming again from the three project,

Output 1.2: Growth-oriented women entrepreneurs are equipped to expand sustainable and flourishing businesses and establish strong cooperatives Output weighting: 50%

· 500 entrepreneurs of which 400 GOWEs and 100 men entrepreneurs have been identified and interviewed as part of the sectoral WED assessment.

2.2.3 Renewable Energy Skills Training

The project secured Private Sector Contribution: USD 1,740,000. (58%) against SIDA’s contributing of USD 2,984,398 for this component and achieved the following thus far;

Outcome 1: IPPs in Somalia participate in a PPDP to produce a supply of skilled workers able to support expansion of renewable energy supply

Output 1.1: Basic training curriculum & certification developed and implemented Output weighting: 35%

· Activity 1.1.1: the project set up technical working group. Key members are selected IPPs and project developers active in Somalia. Specifically, the key implementing partners BECO (Mogadishu), NECSOM (Garowe) and Solargen/CERC (Mogadishu) solar project developer SECCCO (Garowe). Members of the government (including relevant parties from the Ministry of Labour and Social Affairs, Ministry of Education and Ministry of Energy and Water Resources were all involved.

· Activity 1.1.2: with the technical backstopping from the ILO’s Decent work county team in Cairo and the skills department in Geneva, Africa Energy Analytics (EA)2 and Strathmore Energy Research Centre (SERC) were engaged to review the integration of basic curriculum – several drafts of the curriculum were reviewed in the technical working group meetings. Final Curriculum was endorsed.

· Activity 1.1.3: assessment and certification tools were developed. These were inputted and reviewed by the technical working group meeting.

· Activity 1.1.4: The identified training centres (BECO and NECSOM) were evaluated, the missing equipment and other training needs were identified. The equipment list is shared with BECO and NECSOM for their endorsements and equipment are under procurement process.

Outcome 2: A PPDP supported capacity for producing skills specific to the renewable energy sector is established and operational.

Output 2.1: Specialised training programme for solar PV developed and implemented Output weighting: 35%

· Activity 1.2.2: Identification of short-list of key training workshops within IPP coordination committee is fully competed.

· Activity 1.2.3: The identified training centres (BECO and NECSOM) were evaluated, the missing equipment and other training needs were identified. The equipment list is shared with BECO and NECSOM for their endorsements and equipment are under procurement process.

3. Management arrangements for monitoring and evaluation

Overall responsibility for management of the M&E strategy lies with the Technical Officer in charge of the project. Technical support for the design, implementation and backstopping of the M&E functions is provided by Somalia-based project team (M&E Assistant and National Project Officer). These Somalia-based staff are responsible for the supervision of data collection and reviewing the assessment report within the scope of the project.

To the extent possible, the project carries out meetings (Technical working group and project steering committee) with the concerned stakeholders for obtaining feedback on implemented activities and generating lessons learnt for future planning. Further analysis of the reported results is done by the Technical Officer, M&E Assistant, National Project Officer and the ILO specialists. The admin and finance assistant conducts financial monitoring. This allows for a timely update of the work plan and provides the basis for the donor reporting later on.

4. Evaluability assessment Objectives, Criteria, and Clients

4.1 Objectives

• The evaluability assessment will assess the project M&E strategy developed to achieve three main objectives:
• Determine the extent to which the M&E components of ILO have been designed in a manner that will allow for valid and impartial assessment of programme performance, providing information to support programme management decisions and stakeholder reporting needs.
• Provide recommendations for improvements to the M&E strategy that should be implemented by the project staff to improve the programme’s evaluability.
• Identify emergent good practices and lessons learned from the project M&E strategy

  **

4.2 Criteria

The following set of key criteria should be applied in determining the evaluability of the programme:

Criteria

1. Objectives/Outcomes: Clarity of the definition of objectives, including outcomes that can be comprehended as a major focus of management for results
2. Indicators: The selection of SMART indicators that are quantitative and/or qualitative and/or mix and include comparison points of levels, quality and grade. Outcome indicators effectively facilitate the observation of change, while output indicators measure specific deliverables in terms of quantity and quality.
3. Baseline: The existence of sufficient baseline data to establish a starting point for comparisons and future measurements of outputs and outcomes.
4. Milestones: A set of time-bound milestones that provide a clear sense of the intended path towards achieving established outputs and outcomes
5. Risks and assumptions: Assessment of factors, namely risks and assumptions, likely to affect the achievement of an intervention’s objectives, and related contingency measures.
6. Monitoring and evaluation strategy: The M&E strategy identifies problems during project and programme, facilitates the measurement of progress, and enhances the use of the M&E data and information for management, learning, reporting and contribute on the knowledge base on the project themes. The M&E strategy includes a solid and feasible methodology and tools to generate evidence to assess project results based on a Theory of change approach.
7. Gender equality and non-discrimination: Extent to which valid methods for assessing gender differences within the results of the intervention have been developed.
8. SDGs: The project can report its contribution to the relevant SDGs indicators/targets.

4.3 Clients

• The primary end users of the findings of this assessment will be the management team of PPDP, and the ILO’s relevant technical units, the national stakeholders (in support to project implementation, management and learning and to develop their capacities in M&E), and the donor.

5. Evaluation Questions

The evaluability assessment will seek to answer the key questions listed below as measures of the programme’s evaluability and propose practical recommendations including tools, methodologies and activities to address the M&E strategy gaps.

5.1 Theory of change

· Is the project Theory of change consistent and realistic

· Does the Theory of change cover all necessary elements from outputs to outcomes and impact and external factors?

• 5.2 Outcomes
• · Are the result statements specific and easy to understand by anyone?

· Are the results statements for the project clearly articulated?

· How realistic is achievement of the long-term outcomes of the project based upon the outputs and activities proposed?

5.3 Indicators

• · Are the performance indicators (quantitative, qualitative or mix ones) established for the project SMART?

· Do the performance indicators make use of a robust mix of both qualitative and quantitative methods in assessing results?

· Is the collection of data required of stakeholders to measure indicators feasible given their capacities?

• 5.4 Baselines

· Have baselines been established for each outcome indicator?

· Does the baseline establish a credible assessment of the conditions at the start of the programme?

· Does the baseline provide a valid means for assessing the causality of the changes identified (attribution or specific/unique contribution)?

5.5 Milestones

• · Do the milestones provide a clear sense of the timeframe for achievement of results?

· Are the milestones a useful directive for work planning?

· Are the milestones that have been established realistically achievable?

*

5.6 Risks and assumptions

· Have the risks to achieving outcomes been comprehensively identified?

· Are the risk mitigation measures clearly defined and supported by theory, logic, empirical evidence and/or past ILO experience?

· Does the risk management plan provide for regular assessments and adjustments to be made?

· Have the key assumptions to achieving outcomes been agreed upon with stakeholders?

5.7 Monitoring and evaluation strategy

· Does the M&E strategy provide for an adequate quality and regularity of information on programme performance to meet management and stakeholder needs?

· Is the resourcing for the M&E strategy sufficient to meet its objectives?

· Have the criteria for evaluating the results of the programme been clearly defined?

· Are participatory methods in place that will allow for beneficiaries to contribute and use the assessment of programme performance to analysis and learning?

· Does the performance framework provide for a comprehensive assessment of the results of the programme that will allow for informed adjustments to activities with all implementing partners and other relevant stakeholders (such as the beneficiaries)?

· Does the performance framework provide for a comprehensive assessment of the results of the programme that will allow for informed adjustments to activities with all implementing partners?

· Has the project an M&E strategy framework and operational manual for project staff and stakeholders that indicates purposes, methodology, tools and other relevant items to implement it according to its purposes?

5.8 Gender equality and non-discrimination

· How thoroughly hve gender and non-discrimination issues been reflected into the M&E strategy?

· Is data sufficiently disaggregated to analyse gender and other relevant categories differences such as people living with disabilities and inform programme management decisions?

5.9 SDGs, UN Somalia and national development frameworks

· To what extent are there linkages of the results and indicators with the relevant Sustainable Development Goals

· To what extent can the project report on its contribution to the relevant SDGs indicators and targets, the UN Somalia and national development frameworks

6. Methodology of the evaluability assessment

The ILO Policy guidelines for results-based evaluation and specially the EVAL Guidance note on evaluability review will provide the framework for carrying out the evaluability assessment. The evaluation is to be carried out by an independent consultant.

The assignment will be conducted remotely. The evaluator will closely interact with the project team, the main backstopping specialist, and other relevant ILO Staff via teleconferencing facilities such as skype. He/she will be provided with any supporting documents as required, including the project document and subsequent revisions, the project work plan, the progress reports. It is estimated that this work will require a total of 10 working days including review of the final report following ILO comments.

**

7. Evaluation Process and deliverables

The consultancy deliverables expected are as follows

· Inception report

· Draft evaluability report

· Submission of the final assessment report integrating ILO comments and revision.

Any substantial changes from the terms of reference for the assessment will need to be approved by the project Technical Officer.

The process will include:

1. Inception phase 2 days
2. Initial review of project documents on planning and M&E and strategic policy documents (at national level and ILO ones) to be provided by the project
3. Initial interviews with the project Technical Officer, other ILO relevant units for expectations and identify available information required by the consultant
4. Development of the Inception report to operationalize the ToRs
5. Evaluability assessment process 10 days
6. Review of the PRODOC, Project framework, all M&E tools, reports’ templates required, etc.
7. Interviews with the project staff, ILO relevant staff, and relevant stakeholders
8. Development of the draft report
9. Development of the final report
10. Project to collect comments to the draft report
11. Final report.

The proposed list of stakeholders to interview include:

· Project team (Technical Officer, National Project Officer, M&E Assistant and Admin and Finance Assistant)

· Regional M&E officer

· CO Officer based in Addis

· Women Economic Empowerment and Renewable Energy components backstopping in Geneva and Cairo respectively

· Project stakeholders including Federal government of Somalia’s Ministry of Labour and Social Affairs, Ministry of Energy and Water Resources, Somali Chamber of Commerce and Industry (SCCI) and Federation of Somali Trade Unions (FESTU), Banadir Electricity Company (BECO), National Energy Corporation of Somalia (NECSOM) Shuraako, ASAL, Strathmore (Strathmore Energy Research Centre – SERC), KPLC (Institute of Energy Studies and Research – IESR)

· SIDA

Initial list of document to be reviewed

· PRODOC

· ILO evaluation guidelines

· Project reports

· Technical Working Groups meeting minutes

· Steering Committee meeting minutes

The main output of the exercise will be a report assessing the project’s evaluability and proposing practical recommendations and tools. The report should be no longer than 20 pages (excluding appendices). The evaluator will propose an outline for the report as part of the inception report. The Inception report should be approved by the project Technical Officer before proceeding to initiate the actual assessment.

The evaluator will develop a draft report that will be reviewed by the project team. Upon the project comments, the consultant will revise the report and submit a finalized report.

8. Management Arrangements and Work Plan

*8.1 Roles and responsibilities

The EA is managed by the project with technical support from the Regional M&E Officer from the RPU of the ILO Regional Office for Africa. The consultant will report to the project Technical Officer based in Mogadishu, Somalia.

8.2 Consultant profile:

Education

· Advanced degree in relevant discipline (M&E, or technical topic such as development and social studies, sociology, political science, etc

Professional experience:

· At least 8 years’ experience in evaluation in a development context and proven accomplishment in undertaking evaluations, including leading evaluations of multi-stakeholder programmes for multilateral organizations.

· Experience conducting Evaluability Assessments is highly desirable.

· Experience working with UN or other international development organizations and donor agencies would be an advantage

Skills

· Knowledge of the methodology of conducting Evaluability Assessments for ILO projects/programmes

· Knowledge in results-based programming

· Proven expertise in evaluating programmes focusing on labour rights, migration, or another relevant topic

· Extensive knowledge of qualitative and quantitative evaluation methods

· Excellent written and spoken English and presentational capacities

· Excellent inter-personal skills and communication skills

Knowledge of the development context of either Middle East or Africa would be desirable.

PPDP staff: The project will provide information and logistical support to the evaluator during the EA. The project team will also ensure that all relevant documentation is up-to-date and provided to the evaluator.

• 8.3 Work plan

The assessment will take place during April-May, 2020.

9. Deliverables and Terms of Payment

The expected deliverables of the consultant are:

1. Inception Report (20% payment)
2. Draft Report (30% payment)
3. Final Report (40% payment)

This work will require the equivalent of 10 workdays and the payment will be done as a lump-sum (100% of contract fees) upon the submission of the deliverables stated above to the satisfaction of the ILO.

10. Annex-1 – Project Logical Framework**

[1] http://www.oecd.org/dataoecd/29/21/2754804.pdf