External Impact Evaluation – Initiative for Stabilization, Transition, And National Development (I-STAND) Programme

  • Location:
  • Salary:
  • Job type:
  • Posted:
    4 weeks ago
  • Category:
    Monitoring and Evaluation
  • Deadline:

The Nordic International Support Foundation (NIS) is seeking proposals from international organisations/ consulting firms interested in conducting an external impact evaluation of the Initiative for Stabilization, Transition, And National Development (I-STAND) Programme for the implementation period of July 2019 to October 2022 in various Somali regions and communities. The assignment will be completed with the production of an Impact Evaluation report that will inform and guide the partners (NIS, donor, and Somali authorities) in anticipation of the development of the next steps in stabilisation priorities. Please refer to the Terms of Reference (ToR) below for detailed information about the external impact evaluation assignment. Proposal format Interested organisations/consulting firms are required to provide the following details in their application document: • Information about the organisation’s/consulting firm’s background and relevant expertise/experience in program evaluation and conducting third-party monitoring activities (max 2 pages); • List of similar previous services provided and 2-3 samples of previous study reports; • Understanding of the ToR (max 2 pages); • Description of the approach and methodology suggested and an implementation plan with timelines in response to the ToR (max 8 pages); • Detailed financial proposal; and • List of proposed key professional personnel, indicating expected roles and level of engagement (attach CVs).


  1. BACKGROUND With the aim to restore a climate of order in which the Federal Government of Somalia (FGS) and Federal Member States (FMS) can function effectively, NIS has been implementing stabilization projects across Somalia. The I-STAND programme is designed to support a range of government policies and strategies directed at achieving longer-term stability for the country through quick-impact and early-recovery stabilisation interventions, providing basic public services and infrastructure to the communities in newly recovered, fragile, remote, and underserved locations. Some of these interventions target the needs of women and youth in specific. As a result of this provision of infrastructures, I-STAND aims to benefit Somali authorities at the local, state, and national levels of government, by addressing the low level of trust Somalis have in their government and the capacity constraints that limit the government’s ability to meet the needs of its citizens. The programme’s results framework is structured in line with the programme’s Theory of Change which posits that using the delivery of immediate and tangible basic services to strengthen the linkages between the local authorities and communities positively affects both the target communities’ social, economic, security and safety conditions and the targeted authorities’ capacity to deliver basic services. The nature of the I-STAND interventions are categorized in five different modules: Green Module: This module comprises interventions related to the introduction and usage of renewable energy and protection of the environment. By using solar-based interventions, I-STAND intended to ensure that conflict-affected populations experience tangible benefits from projects immediately. For example, the installation of solar streetlights aimed at increasing security, economic opportunities, mobility, and social interaction/cohesion beyond daylight hours for both men and women. The solar electrification of social institutions, hospitals and government buildings aimed at supporting the governments capacity to provide social services and increase their visibility to the communities. As it stands, the dissemination of green technology in Somalia through these types of projects is helping to attract private investments in the sector, as well as contributing to new demands/opportunities in the labour market. Jobs Module: Interventions under this module focused on offering employment opportunities in beneficiary communities through, on one hand, labour-intensive interventions (incl. road construction, brush clearing, rehabilitation and upgrading of long floods prevention embankment and rehabilitation/ upgrading of communal water catchment infrastructure) and, on the other hand, vocational training, both of which hoped to result in community recovery (through infrastructure delivery) and improved livelihoods of workers and their families. These interventions also sought to enhance the rule of law as security improves through the engagement of youth in income-generating activities. Cohesion Module: Through the rehabilitation of sports infrastructure for male and female youth and the delivery of New Engagement Through Sports (NETS) activities, including sports tournaments in the new facilities, I-STAND projected for beneficiary communities to achieve greater social cohesion as they create a sense of shared identity and fellowship through these types of activities. However, the programme also takes into consideration that other of its interventions may also have an impact on social cohesion, such as market and community centre construction and the installation of solar streetlights. Additionally, throughout the planning and implementation phases of the 3 interventions, community ownership is encouraged, which is believed to sustains peace as communities jointly protect these common assets, thus preventing conflicts. Brick & Mortar Module: Interventions considered under this module are related to the rehabilitation and construction of physical infrastructure (incl. roads, communal and government buildings such as schools, markets, bridges and community centres) with the purpose of strengthening trust in government through leadership of its officials in community recovery and stabilisation efforts and their visibility. Think Tank Module: This module entails knowledge generating and information leverage activities, such as assessments, community engagement and consultations, and strategic communication. By creating community platforms, ensuring equal participation, consensus building and empowering the marginalised, community engagement and consultations create common understanding of community-needs priorities, including those of women, youth and other marginalized groups, and help envision a shared future which consequently aids stabilisation efforts in the country. Through these interventions, NIS evaluates the impact of programme activities, especially the communities’ trust in the government. The module also deepens NIS’s knowledge about various interventions which advance the creativity, adaptability and efficiency in programming to enhance implementation and sustainability of stabilisation projects. The proposed locations, and respective interventions conducted, to be covered by the impact evaluation are specified below: Table 1: Sampled study locations1 Region District Intervention Type Banadir Mogadishu Installation of Solar streetlights Banadir Hospital electrification Renewable energy for small scale companies including women owned business Waste management Microfinancing of women-owned SMEs Mogadishu Stadium Rehabilitation (incl. solar streetlighting) Road rehabilitation/ construction Hiiraan Beledweyne Rehabilitation and upgrading of long floods prevention embankment & solar streetlighting Lower Shabelle Janaale Installation of Solar Street lighting Construction of market Road rehabilitation Inc. Repairs of strategic culverts Brush clearance Rehabilitation of gravel road from KM50 Construction and installations work of the baile bridge Mudug Galkayo South Galkayo hospital electrification 1 Sample locations may be amended in the inception phase 4
  2. PURPOSE AND OBJECTIVES OF THE EXTERNAL IMPACT EVALUATION The overall evaluation process will be based on the OECD/DAC criteria2. Following these criteria, the purpose of the evaluation is to explore the relevance, effectiveness, efficiency, impact, coherence/complementarity, and sustainability of the programme and to provide NIS and other stakeholders (including the donor and the government) detailed findings and recommendations to improve the quality of its operations. The evaluation exercise shall further examine the underlying assumptions and weakest links of the programme’s ToC, challenges encountered during the programme implementation in the specific target areas and the lessons learned. 2.1 Specific objectives of the study The study is intended to: ➢ Assess the positive or negative, intended or unintended, effects and impacts of each type of the programme interventions on the community, considering specific segments of the community such as minority groups, women and youth and the geographic distinction between frontline (newly liberated areas) and strategic locations, which is central to I-STAND’s intervention logic. ➢ Provide an analysis of the assumptions, found in the RF, underlying the programme’s overall Theory of Change (ToC), identifying weaknesses and strengths, and make recommendations for adjustments to improve the programme’s ToC to build confidence in the government through the provision of basic services to communities. ➢ Provide an analysis of best practices and effectiveness of the implementation of ISTAND, taking into account challenges/lessons learned of stabilisation interventions (what worked well and what did not in previous/ ongoing interventions?). Based on the programme goals and the goals of the National Stabilisation Strategy specifically, highlight where support positively impacted state building, focusing on projects that supported the core functions of government/states through institution building and payment of salaries (capacity building). The analysis and findings should inform strategies and approaches for future programming. ➢ Referencing the NIS Somalia GESI Strategy, undertake a detailed analysis of gender and inclusion dynamics in the political, economic, security/conflict contexts all the proposed target locations. After doing this, propose strategies to enhance gender inclusion and integration in these contexts to optimize the programme outcomes for women, youth, and the marginalized groups in the Somali context. The impact evaluation report should assess the programme’s overall relevance, effectiveness, efficiency, coherence and complementarity with other stabilisation actors, sustainability, and impact, specifically addressing the following:
  3. Determine the extent to which the objectives (outputs, outcomes, and impact) of the programme as set in the programme proposal were achieved;
  4. Assess the relevance and solidity of the Theory of Change and the related assumptions in the results framework. Identify contributing factors that led to both the achievement and/or failure of achieving the outcomes and how the programme managed or dealt with them;
  5. Assess the degree to which systems were put in place by I-STAND for ensuring the sustainability of the specific projects upon completion and their current status; 2 From the OECD-DAC Criteria for International development evaluation – website http://www.oecd.org/dac/evaluation/daccriteriaforevaluatingdevelopmentassistance.htm 5
  6. Evaluate the quality of involvement of the target groups i.e., community steering committees and the quality of the broader community engagement;
  7. Assess the extent to which programme activities contributed, or if they did at all, to the promotion of human and women’s rights, specifically the economic empowerment and inclusion of women in the labour force;
  8. Appraise the extent to which the programme and its components are replicable elsewhere in Somalia;
  9. Consider the performance of NIS and its partners in obtaining value for money and achieving economies of scale in relation to other stabilisation actors; and
  10. Assess the extent to which the programme succeeded in achieving VfM in terms of: o Economy – Procuring the best-value inputs; o Efficiency – Maximising the outputs for the given level of inputs of the different modules; o Effectiveness – Ensuring that outputs achieved the desired outcomes in the best way possible o Equity – Ensuring the equitable distribution of programme benefits the most vulnerable groups.
  11. Discuss other possible interventions taking the aspect of cost efficiency into consideration.
  12. TECHNICAL APPROACH AND METHODOLOGY The Consultant should employ a mixed-method approach, combining qualitative and quantitative data collection methods. The Consultant also must specify the methods to be used in data collection processes to achieve the deliverables efficiently and effectively. In the technical proposal, the consultant is expected to specify appropriate technical approaches and methodology, and details for each method, including the types of questions to be asked; intended profile (including gender) of participants; timeline of field activities; sources to be used for secondary desk review, etc. The Consultant should conduct an independent evaluation using data collection tools chosen by the Consultant and outlined in the proposal. The evaluation will be conducted in accordance with recognised evaluation principles and the OECD DAC’s quality standards for development evaluation and guidelines for evaluations in fragile and conflict settings. The impact assessment will be carried out in all the sampled target locations/districts (ref. Table 1). The following data collection strategies could be applied by the Consultant (additional methods may be applied, subject to approval by NIS): ▪ Conduct a comprehensive desk review of all programme documents that will be provided by NIS, and any other relevant literature prior to tool development. ▪ Develop quantitative and qualitative data collection tools for the impact evaluation. ▪ Conduct quantitative household surveys using digital data gathering software (ODK/ONA). Gender balance will be maintained during data collection by hiring both male and female enumerators, which is expected to ensure inclusion and encourage women to participate in interviews and surveys. Enumerators will be properly trained by the Consultant, and a pilot study will be conducted prior to the extensive survey to verify that questionnaires are clearly understood. ▪ Carry out economic activity mapping (gender disaggregated) which will consist of counting the businesses near where solar streetlights have been installed and interviewing 20% of the business owners regarding their income, rent costs, and hours of operation (comparing before and after installation of solar streetlights). 6 ▪ Conduct Key Informant Interviews (KIIs) and Focus Group Discussions (FGDs) with a series of programme stakeholders, such as local, federal, and regional authorities, community leaders, local business owners, women’s representatives, youth’s representatives, hospital staff, security officers, implementing partners, vocational training trainers and students, sports officials and players and labourers engaged in labour-intensive activities, and record observations and checklists ▪ Gather stories of change, depicting how the programme has affected the lives of the communities living in the targeted locations. The most significant change method is preferred but other methodologies can be suggested.
  13. DELIVERABLES The required outputs of this assignment will be as follows:
  14. Inception report including a work plan, detailed methodology, report format and timeline is to be submitted at the commencement of the assignment. The inception report will include the following and a debrief meeting will follow the report to clarify some of the deliverables, budget, timelines etc: – Proposed methodological approach that the consultant will employ; – All quantitative and qualitative data collection tools for review and approval; – Complete work plan with clear timelines (including researchers’ level of efforts, sequencing of impact evaluation activities and data collection and the budget cost of the assignment) – Limitations of the data collection tools – Data quality assurance mechanism – Report format
  15. Raw data for each data collection exercise compiled in electronic format: – Household/ hospital patient survey data submitted to NIS’ digital data collection platform ONA; – List of businesses counted in the Economic Activity Mapping in Excel or Word; – Economic Activity Mapping (business owner surveys) data submitted to ONA; and – All data from KIIs and FGDs transcribed and translated to English.
  16. Full sets of photos collected, including captions detailing the dates, locations, and the project site visited. Where possible the research assistant should provide some more background information on each set of photos.
  17. Prior to impact evaluation report submission, the consultant is expected to make a presentation of the main findings of the evaluation to NIS MEL and programme team and the donor for review, comments and discussion.
  18. Presentation of the final version of the impact report to programme team, external programme stakeholders, RNE and other stabilization donors.
  19. Overall programme impact report, not exceeding 30 pages, that includes values for the programme indicators as outlined in the results framework.
  20. At least 3 stories of change (at least one of a female beneficiary) showing how the programme impacted on the target communities. 7
  21. TIMELINE The engagement of the Consultant for the impact evaluation is expected to be for a period of not more than 90 days for the developing of the workplan, the methodology, the tools, the collection of data, the analysis and reporting. This is effective from the day of signing the contract by the parties and subject to the approval of final impact evaluation report from both NIS and the donor. The technical proposal shall include a clear proposed timeline and level of engagement of lead consultant and assessment team.
  22. SUPERVISION AND IMPLEMENTATION ARRANGEMENTS The Consultant will – Consultant will work independently but maintain constant communication with NIS’ Monitoring, Evaluation & Learning Manager for any clarifications – accomplish his/her responsibilities in close coordination with NIS and deliver the outputs as described above. NIS, in consultation with the donor, as necessary, will – assure the quality of reporting through timely reviews, feedback, inputs and requesting additional information as necessary. – take an active part in working with the Consultant to develop necessary impact evaluation tools for data collection and reporting etc.; – provide programme documents, reviews, assessments and other relevant materials for desk review; – provide guidance in mapping the assessment sites; – provide necessary contacts of stakeholders and beneficiaries.
  23. REQUIRED QUALIFICATIONS AND EXPERIENCE OF THE CONSULTANT/S The lead consultants must demonstrate the following qualifications and expertise: ▪ Advanced university degree in monitoring and evaluation, statistics, political science, rule of law, security, conflict management/ resolution, international relations, or other relevant field from the team involved; ▪ A minimum of 10 years of relevant/ similar professional experience in conflict/post-conflict settings, preferably in the Horn of Africa region, and/ or Somali speaking areas; ▪ Proven experience in conducting similar research/evaluation activities and producing quality analytical reports for international organizations and donors; ▪ Fluent in English and Somali languages and good knowledge of the Somalia political, security and socio-economic context/situation; ▪ Previous experience working with Somali institutions and communities and the ability to work in and lead a team of surveyors in relevant programme target areas; ▪ Capacity and proven experience in collaborating with diverse sets of stakeholders, such as civil society organizations, community representatives, government officials, donor representatives, international institutions. 8
  24. AWARD CRITERIA NIS will evaluate proposals and award the contract based on technical and financial feasibility3 and reserves the right to accept or reject any proposal received without explanation and is not bound to accept the lowest bidder. Bidders who fail to provide valid legal documentations (e.g., registrations, valid tax compliance certificates, supplementary certificate of courses, institutes etc.) will not be considered for this consultancy. Award of the contract will be based on a combination of the following criteria:
  25. Understanding of the assignment and experience/ expertise of the consultant relevant to the assignment: • Demonstration of clear interpretation of the TOR, study design and required qualifications and their relevance to the assignment: 15 marks • Key expertise required for the assignment for the lead consultant/s and the assessment team (as per ToR, Point 7 above): 15 marks Subtotal: 30 marks
  26. Adequacy and quality of the proposed methodology and work plan in achieving objectives and deliverables outlined in the ToR: • Technical approach and methodology and clarity of data collection tools; and proper integration of gender and inclusion dynamics in the assessment areas/components: 15 marks • Work plan and proposed activities timelines: 5 marks • Understanding of and demonstrated experience operating in insecure environment: 5 marks Subtotal: 25 marks
  27. Understanding of the context • Proven experience in conducting impact evaluation studies in stabilization context in Somalia: 15 marks • Demonstration of mobility and access to programme areas: 5 marks Subtotal: 20 marks
  28. References • References from similar work done elsewhere (2-3 samples of similar reports needed): 5 marks • Demonstrated Value for Money of previous assignments: 5 marks Subtotal: 10 marks
  29. Financial proposal (competitiveness of the vendor) • Price and Value for Money: 10 marks • Willingness to negotiate: 5 marks Subtotal: 15 marks Grand total marks: 100

How to apply

Submission of proposals The deadline for submitting complete proposals is 16:00h on 14 November 2022. Proposals must be submitted electronically to procurement.somalia@nis-foundation.org, with the subject line “NIS External IMPACT Evaluation – Initiative for Stabilization, Transition, And National Development (I-STAND) Programme in Somalia – CALL FOR PROPOSALS”. Contact information Please forward any query to the following e-mail address: procurement.somalia@nis-foundation.org.

This job has expired.