Evaluation Terms of Reference

ANCP-FMNR PHASE II PROJECT

World Vision Somalia and

World Vision Australia

1. Project Details

Project Name

Enhancing Resilience for Improved Livelihoods in Odweyne and Burao Districts in Togdheer Region and Cel afweyne district in Sanag region

Project Number

PBAS # 208849

Country and district(s)

Somaliland, Odweyne, Burao and Cel afweyne districts

Start and End date of Project

:01 July 2017

Date Baseline data completed

March 2018

Total Project Budget

USD2,340,600

Source of funding:

Australian Government Department of Foreign Affairs and Trade through Australian National NGOs Cooperation Programme (ANCP)

Estimated Beneficiaries (Direct)

Total Households:

Total people:

Men:

Women:

Girls:

Boys:

:2241

: 13,444

6719

5784

480

461

Contents

1. Project Details. 1
2. Project Background. 4

The objectives of the project are: 5

Outcome and output indicators: 6

1. Purpose the Evaluation. 7
2. Evaluation Objectives. 7
3. Guiding Evaluation Questions. 8
4. Evaluation Leaders. 9
5. Travel expectations of lead evaluator and security context of location. 11
6. Proposed Evaluation Products and Milestone Dates. 12
7. Lead Evaluator Profile. 12
8. Evaluation Report Outline. 13
9. Anticipated Limitations. 15
10. Budget 16
11. Standards of Ethics and Child Protection. 16
12. Documents to be made available for evaluation preparation. 17

GLOSSARY

ANCP Australian NGO Cooperation Programme

AP Area Programme

CBO Community Based Organisation

CED Community Economic Development

DME Design Monitoring and Evaluation

DPA World Vision Development Programme Approach

ECCD Early Childhood Care and Development

FGD/ Focus Group Discussions

KII Key informant interview

MNCHN Maternal, Newborn Child Health and Nutrition

PLA Participatory Learning and Action

TOR Terms of Reference

WASH Water, Sanitation and Hygiene

WVA World Vision Australia

2. Project Background

This project aimed to address the vicious cycle of detrimental effects of climate change such as recurring droughts, negative coping mechanisms adopted by communities, degradation of natural resources, increased vulnerability and diminished resilience of local communities in Odweyne, Burao and Cel afweyne districts. The project aimed to enhance the resilience of the targeted agro-pastoralist communities through improved eco-system and food security

The project sought to increase smallholder farmers’ production and productivity, and therefore their food security, through strengthening their livelihood assets. This included interventions at both household and community level. At the household level, WVS aimed to restock livestock and provided training to communities to increase their production capacity as well as create awareness in the preservation of the environment. At the community level, the project strengthened the veterinary service, improve livestock and water infrastructure and improve grazing land and farmland in the communities, aiming to create an enabling environment for sustainable livelihood development

The project also focused on diversifying the livelihood activities to improve the opportunities for vulnerable households to access additional income from off-farm activities, which are less climate-sensitive. The livelihood activities included bee-keeping, poultry production and block making. To increase communities’ access to financial resource that can either cushion them in a time of stress or allow business expansion, the project facilitated the establishment of savings groups.

Improving natural resource assets that yield sustainable livelihood outcomes is the major objective of this project. Both on-farm and off-farm natural resources were to be covered in the project, which included rehabilitation of irrigation system, community participation in natural resources management for sustained livelihoods, rangeland and pasture rehabilitation and soil and water conservation. These activities took the Farmer-managed natural regeneration approach (FMNR) to systematically regenerate and manage natural resource. FMNR sites were used as learning centres for agro-pastoralists and partner research institutions to continuously improve and further promote this low-cost and sustainable natural resource management technique to yield wider impact beyond the targeted locations. The FMNR and environmental protection activities implemented in the target villages involved faith-based and/ or community-based organizations in educating targeted communities on the effect of deforestation and land degradation. The project built on the learning from the previous pilot project in Burao district on FMNR to extend the approach to the new project locations in Odweyne and Cel-afweyne districts.

The three-year project targeted 13,444 direct participants (7,124 men, 6,240 women, 56 men with disability and 24 women with disability). An approximate of 80,664 communities’ members were indirectly targeted to benefit.

Justification

Recurring natural disasters and negative coping mechanism

Somaliland is an unrecognized de facto sovereign state situated in the north-west of Somalia. Despite positive developments, its government institutions remain weak, and therefore cannot adequately provide public services and build the local capacity to address the challenges of climate change such as frequent droughts, and limited livelihood diversification opportunities. Consequently, recurrent drought, livestock diseases and environmental degradation have persistently and adversely affected the livelihoods of communities in Somaliland. As land and pasture are degraded, crop and livestock productivity declines, resulting in decreased income security and threatening the sustainability of the livelihoods of communities, especially the most vulnerable one. This leads to migration specifically among males who would have no choice but to leave their home in search of other income opportunities, leaving women and children behind in rural areas with little or no livelihood assets to support them.

Climate sensitive livelihoods and missing livelihood opportunities

Agro-pastoralists rely heavily on good climate condition to sustain their livelihood. Yet, in places like Somalia where drought is cyclical and frequent, climate-sensitive livelihoods give communities limited ability to adapt and absorb the recurring shocks if they focus only on a certain group of crops. Improving the existing livelihood opportunities and diversifying means of living is therefore important in building resilience of communities.

Invasive prosopis juliflora

Prosopis juliflora, locally known in Somali as Ali Garoob, is an evergreen tree native to South America, Central America and the Caribbean. In the United States, it is well known as mesquite. It is fast-growing, nitrogen-fixing and tolerant to arid conditions and saline soils. Prosopis was introduced to the Horn of Africa without preliminary long term impact studies. AFRICARE (1983)[1] reported the introduction of Prosopis juliflora in Somalia as part of a reforestation project for sand dune stabilization in a refugee impacted areas in the Hiran region of central Somalia during the pre-conflict period, and it spread uncontrollably at alarming rate during the conflict period. The tree has survived where other tree species have failed and, in many cases, has become invasive and a major nuisance to the environment. The species crosses borders and affects the entire ecosystems and natural resources upon which the poor and hard-pressed pastoral and agro-pastoral communities depend for their livelihoods

The objectives of the project are:

Project Goal: Enhancing resilience through improved ecosystem health and food security of agro-pastoralist communities in Odweyne and Burao Districts in Togdheer Region and Cel afweyne district in Sanaag region

Outcome 1: Increased production and productivity of smallholder farmers and agro-pastoralist

Outcome 2: Increased capacity for targeted communities to diversify and expand their livelihood opportunity

Outcome 3: Improved natural resource assets supporting small-holder farmers and pastoralists

Outcome 4: Enhanced community resilience and disaster mitigation capacity

Outcome and output indicators:

Project Goal: Enhancing resilience through improved ecosystem health and food security of agro-pastoralist communities in Odweyne and Burao Districts in Togdheer Region and Cel afweyne district in Sanaag region

Outcome 1: Increased production and productivity of smallholder farmers and agro-pastoralist

Indicators: Percentage increase in production by smallholder and agro-pastoralist households

Proportion of households with increased income levels

Proportion of households with increased yield

The outputs indicator listed below were key contributors to the above outcome the respective indicators; # of households practising crop and livestock husbandry practice and # of water infrastructures developed /rehabilitated. The activities under outcome one focused on empowering both the demand (communities) and the supply side (service providers), particularly extension workers with skills and inputs for increased production and productivity.

Outcome 2: Diversified livelihood alternatives among the targeted agro-pastoralists

Indicators: Proportion of households where one or more adults are earning an income from selected enterprises

The output indicators contributing to outcome 2 included; # of households where one or more adults are earning an income from selected agricultural enterprise, and # of savings groups established and are functional. The focus of activities on outcome 2 was to build capacity and skills of targeted communities to embark on difference enterprises, both on and off-farm. The empowerment was aimed at enabling households to realise more income from multiple enterprises.

Outcome 3: Improved natural resource assets supporting small-holder farmers and pastoralists

Indicators: % increase in tree, grass and legumes density in hectares

The key outputs contributing to outcome 3 were; # of hectors of farming land been rehabilitated and used, and # of trees propagated and planted off-farm. Activities under this outcome and respective outputs focused on rehabilitation of and sustainable use of available natural resources.

Outcome 4: Enhanced community resilience and disaster mitigation

Indicators: # of Ministry of Agriculture, NERAD staff trained and with capacity to operationalise early warning and early action

This outcome was linked with the following outputs; # of disaster risk reduction structures formed and functional in the community. Community early warning sms platform developed and functional, # of households that can employ coping strategies to mitigate disasters and # of people trained and are aware of early warning systems in times of shocks.

ANCP MELF Indicators: Mandatory Indicators

Code Indicator

2.802 – Number (x) of local in-country partners that collect disability-disaggregated data

6.101 – Number (x) of local in-country partners participating in ANCP funded projects/programs

6.103 – Number (x) of local in-country partners reporting an increased organisational and project implementation capacity as a result of participating in ANCP funded projects/programs.

ECONOMIC DEVELOPMENT

Number

Indicator

Instructions

Project # 208849

FOOD SECURITY AND RURAL DEVELOPMENT

3.101

Number (x) of poor women and men who adopt innovative agricultural and fisheries practices

Yes

3.102

Number (x) of poor women and men with increased incomes

Yes

3.201

Number (x) of poor women and men who increase their access to financial services

Yes

3.202

Number (x) of additional poor women and men (and where specified boys and girls) able to access social transfers (such as cash or in-kind transfers including food)

Yes.

3.204

Number of people with new waged employment

No

CLIMATE CHANGE

3.306

Number (x) of people provided with climate resilience training

Yes

3.307

Number (x) of local climate change assessments and action plans developed in conjunction with communities

NO

3.308

Number (x) of people with additional sustainable energy access

Yes

3.309

Number (x) of people with a new income source from sustainable environment and natural resource use

Yes

3.310

Number (x) of activities/training for community leaders and organisations which include a focus on climate change and environment issues

Yes

3.311

Number (x) of people exposed to awareness-raising campaigns/activities highlighting climate change and environment issues

Yes

3. Purpose of the Evaluation

This study is an end-of-phase summative outcome evaluation. Results will be used to measure project results against its design objectives and guide the redesign of the second phase of the project lifecycle.

4. Evaluation Objectives

The principal objectives of the evaluation are:

1. Verify that the project has achieved the stated outputs,[2] and outcomes among disaggregated stakeholder groups (including gender and disability)

2. Evaluate the relevance of the intervention and appropriateness of implementation approaches used.

3. Document promising practices, key lessons learned and recommendations, which will refine the project design for a follow on phase of the project

4. Identify weaknesses in the project design, implementation or operating environment that constrained optimal project effectiveness

5. Provide on-the-job learning and training to World Vision and Partner staff in evaluation methods

5. Guiding Evaluation Questions

Relevance

· To what extent did the project design and implementation address perceived and evidenced community vulnerabilities and government priorities?

· To what extent are community members satisfied with the implementation and results of the project?

· How fair and appropriate was the implementation of the project? (E.g. considering gender, youth, culture, disability, displacement, etc.)

· How were power and gender dynamics analyzed and addressed in the project?

· How were women, children (girls & boys), males & females with disability, female-headed HH, youth, male & female IDPs and members of host communities involved in the design, implementation and monitoring?

Effectiveness

· To what extent have output, outcome and goal level of project indicators been achieved?

· What are the achievements against the set targets? Compare the endline results to the baseline.

· Compare actual with planned outputs and how have outputs been translated into outcomes.

· The evaluation will also establish the possible deviation from planned outputs and likely outcomes.

Efficiency

· Were the objectives of the project realistic relative to the funding provided?

· How efficient were the project activities? Were all the project resources utilised optimally?

· How could we have done things more cheaply and on time?

· How could the efficiency of the project be improved without compromising outputs?

· How adequate were the reporting and monitoring systems of the project?

· To what degree did the outcomes justify the costs, and how does this compare with similar projects in terms of return on investment?

Sustainability

· How sustainable are the systems and benefits established by the project?

· How likely are the outcomes to be sustainable and enduring?

· In what ways will it leave a legacy for its beneficiaries and the communities?

· In what ways are women and men in communities, the local partners and government stakeholder’s partners prepared to continue with the project outcome?

Impact

· What positive and negative changes occurred as a result of the project, beyond what was initially planned by the project design?

Cross-cutting themes: To what extent have the following five cross-cutting themes been addressed in actual implementation: disability, gender, peacebuilding and conflict-resolution, protection (including child protection) and environmental stewardship. Examples of issues to address:

· GENDER: To what extent do men and women (and girls and boys) have equal access to project benefits?

· DISABILITY: To what extent do differently able people have meaningful participation and access to project benefits?

· ENVIRONMENT: Did project monitoring and implementation adhere to its own environmental management plan to mitigate negative impacts and optimise positive impacts on the environment?

· PROTECTION: Have project management complied with DFAT Child Protection guidelines to ensure children were not put at risk?

· PEACE-BUILDING: Have programme activities promoted understanding between different groups or created tension?

6. Evaluation Leaders

WV Somalia will establish an evaluation team to oversee all the related tasks. The Quality Assurance and Strategy Manager will be responsible for the overall coordination of all the evaluation tasks with the Consultant. Besides, the Project Manager, Regional Operations Manager and the Technical Unit Lead will provide all the necessary technical and operational support required throughout the evaluation process.

Support from WV Somalia

WV Somalia will be responsible for the following:

§ Recruit the external Consultant and finalise the consultancy agreement

§ Share all necessary documents to the Consultant to finalize the evaluation methodology and data collection tools

§ Provide input for evaluation study methodology, data collection tools and report.

§ Ensure that input from WV Somalia is circulated and shared with external Consultant

• Flight expenses for the Consultant to Somaliland (where necessary)
• Vehicle hire to support the evaluation exercise
• Food and accommodation for the Consultant in Somaliland
• Working space for the Consultant while in Somaliland
• Recruitment and payment of enumerators
• Stationery for data collection

§ Overall accountability of the evaluation process

§ Guidance and coordination throughout all the phases of evaluation, keeping communication with external Consultant throughout all phases

§ Provide support to the evaluation technical lead (external Consultant) for the evaluation field visits processes such as orientation and training of enumerators, FGDs and KIIs

§ Closely follow up the data collection process, ensuring quality control, daily debriefing, meeting the timelines set for interview completion;

§ Inform evaluation audience for their involvement in the study and help in setting specific dates for the evaluation field schedule.

§ Provide smartphones/tablets, ODK aggregate server for data collection where required.

Core Evaluation Team

Name

Role & Expertise

Stage of Design

Specific duties

Lead Evaluator

1. External consultant to be identified

Independent evaluation consultant, specialised in food security, Natural resources management and sustainable livelihoods

Support all stages of Evaluation Process

Review project documents.

Lead development of a detailed evaluation plan.

Develop qualitative and quantitative data collection tools.

Conduct training of enumerators and implement data quality checks.

Collaborate with World Vision Somalia to ensure evaluation is conducted efficiently, within budget, and safely

Oversee and support data collection, especially qualitative activities.

Lead qualitative data collection with key stakeholders.

Complete data analysis with respect to key evaluation questions.

Draft and finalise the baseline/ evaluation report.

World Vision assistant evaluators

1. NAME: WV Somalia Quality Assurance and Strategy Manager

Key Evaluation Facilitator – DME Technical support

ToR development

Selection of Lead evaluator

Pre-data collection logistical arrangements

Deputy for data collection process

Liaison during analysis and writing period.

Write ToR, in consultation.

Organise data collection logistics.

Facilitate Staff Evaluation Training
Lead delegated aspects of data collection.
Coordinate all staff involved in data collection
track, receive and collate all data via data collection framework.

1. NAME: WVA E&L Advisor

[For Priority 1 and 2 evaluations]

Evaluation Lead for Priority 1 evaluations

Evaluation Support for Priority 2 evaluations

Support all stages of the Evaluation process (most likely, in field)

Support all stages of the Evaluation process (most likely, from Australia)

Lead approach to project evaluation.

Write ToR, in consultation with NO.

Select lead evaluator.

Liaise with NO re data collection logistical arrangements.

Oversee data collection process.

Guide consultant in data analysis and reporting.

Review and accept final report.

Provide advice regarding the approach to evaluation.

Advice on selection of consultant.

Review ToR.

Review evaluation plan and data collection tools.

Guide consultant in data analysis and reporting.

Review and accept final report.

1. Charles Otieno, Technical Advisor and Technical Unit Lead

WV Somalia Sector lead

Planning

Data collection

Report draft review

Support development of the ToR, Evaluation Plan, and data collection, technical assurance; staff training on data collection; and lead aspects of data collection.

9. Team Advisors

WV stakeholders/advisors

Name

Role

Stage of Design

Specific duties

Madeleine Bilonda

Operations Manager

Pre-evaluation

Evaluation sponsor.

Scheduling of evaluation into project implementation plans.

Facilitation of project staff and community availability.

Abraham Assefa

WV Som Project Manager

Pre-evaluation

Submission of draft

Assist the development of Terms of reference and Evaluation Plan.

Review report drafts, and provide constructive feedback for improvement.

WVA FSL Sector Advisor

Pre-evaluation

Submission of draft

Assist the development of Terms of reference and Evaluation Plan.

Review report drafts, and provide constructive feedback for improvement

WVA Evaluation Advisor

Pre-evaluation

Submission of draft

Assist the development of Terms of reference and Evaluation Plan.

Review report drafts, and provide constructive feedback for improvement and DFAT compliance.

WVA ANCP Grant manager

Pre-evaluation

Submission of draft

Assist the development of Terms of reference and Evaluation Plan.

Review report drafts, and provide constructive feedback for improvement and DFAT compliance.

7. Travel expectations of lead evaluator and security context of location

The lead evaluator is expected to be available to spend approximately 20 days in the country of the project for data collection preparation, collection and debrief in December and at least before 20/12/2019 (subject to refinement at contract stage).

The evaluation proposed budget will cover the costs of travel visas, transit to and from their designated home point to the project locations, and accommodation, for the duration of the data collection period. The estimated cost of transit will form part of the overall cost consideration for selecting a lead evaluator.

The consultant will be responsible for his or her insurances, vaccinations, health, and security preparedness.

The following proposed dates will be subject to negotiation and revision, with the chosen candidate.

Products

Due date:

Develop Evaluation Design/Plan, including all data collection tools, i.e. FGD and KII guiding questions and PRA exercises:

30^th November 2019

Proposed data collection dates (Subject to adjustment with lead evaluator):

Product will be: All data records whether in soft or hard copy e.g. transcripts, databases, spreadsheets, photographs: Photographs and audio data, by the last day of in-country work. Written and statistical documentation handed to WV Country Office

December 20th

1st complete draft evaluation report:

By 14^th Jan 2020

2^nd complete draft evaluation report, incorporating initial feedback

By 30^th Jan 2020

final version evaluation report, incorporating second round of feedback:

(Final version to include an executive summary that can stand alone as a self-contained summary report)

14 Feb 2020

8. Proposed Evaluation Products and Milestone Dates

The Consultant will be expected to deliver the following outputs:

§ An inception report detailing the approach and methodology to be used and sample size calculations, a detailed execution plan, data-collection tools.

§ Draft report submitted to WV Somalia within an agreed timeline between the WV Somalia and the Consultant (soft-copy)

§ A presentation of the key findings and recommendations to WV Somalia and other stakeholders in Hargeisa (this is optional depending on if the consultant chooses to remain in-country during report write up, however, Consultants able to complete this deliverable will be preferred. Either way, presentation will be required to be made to WV Somalia).

§ All indicators must be presented overall and disaggregated by sex and disability status, where appropriate.

§ An up to date (populated) Indicator Tracking Table (ITT) with all outcome level indicators

§ Collected data (raw) after analysis complete with variable labels and codes, and the final evaluation tools submitted to WV Somalia and alongside the final report.

§ Final report (soft copy) and 3 hard copies submitted to WV Somalia Quality Assurance team and Food Security and Livelihoods Manager. However, the Consultant should note that the Final Evaluation Report shall follow the structure below customised from the UNDP (2009) Handbook on Planning, Monitoring, and Evaluation for Development Results.

9. Lead Evaluator Profile

The lead evaluator must possess the following skills and qualifications:

§ The team leader MUST possess at least a Master’s Degree in any of the following fields; International Development, Social Sciences, Statistics, Community Development, Development Studies, Natural Resource Management or any related field

§ Strong and documented experience in conducting participatory qualitative assessments related to natural resource management, livelihoods and water and sanitation programming.

§ Demonstrated experience in leading at least three similar project evaluation studies such as surveys and group interviews,

§ At least ten years’ experience in conducting baseline and evaluations for complex projects such as natural resource management, infrastructure development, livelihoods, water and sanitation and hygiene being implemented by non-governmental and private sector actors.

§ A solid understanding of remote learning and use of mobile technology in data collection,

§ Demonstrated experience in leading teams, training local staff in quantitative and qualitative data collection tools including entry template

§ Demonstrated experience in designing survey methodology, data collection tools, processing and analysis of data.

§ Ability to interact with host government, partners as requested by WV Somalia;

§ Strong organisational, analytical and reporting skills, presentation skills, attention to detail, ability to meet deadlines, and proficiency in both quantitative and qualitative data analysis software.

§ Previous experience in a fragile country with tight security context will be preferred.

§ Excellent verbal and written communication in English required.

10. Evaluation Report Outline

Title and Opening pages (front matter)-should provide the following necessary information:

i. Name of the project evaluated

ii. Time frame of the evaluation and date of the report

iii. Project location (districts and country)

iv. World Vision logo

v. Acknowledgements

Table of Contents-including boxes, figures, tables, and annexes with page references.

List of acronyms and abbreviations

Executive Summary

A stand-alone section of two to three pages that should:

§ Briefly describe the project that was evaluated.

§ Explain the purpose and objectives of the evaluation, including the audience for the evaluation and the intended uses

§ Describe the key aspect of the evaluation approach and methods.

§ Summarize principle findings, conclusions, and recommendations.

§ Include a summary table displaying the figures (scores) for each indicator at baseline and at end line, with elaboration on the statistical significance of differences.

Introduction

§ Explain why the evaluation was conducted (the purpose), why the intervention is being evaluated at this point in time, and why it addressed the questions it did.

§ Identify the primary audience or users of the evaluation, what they wanted to learn from the evaluation and why and how they are expected to use the evaluation results.

§ Identify the project that was evaluated

§ Acquaint the reader with the structure and contents of the report and how the information contained in the report will meet the purposes of the evaluation and satisfy the information needs of the report’s intended users.

Description of the Intervention

Provide the basis for report users to understand the logic and assess the merits of the evaluation methodology and understand the applicability of the evaluation results. The description needs to provide sufficient detail for the report user to derive meaning from the evaluation. The description should:

§ Describe what is being evaluated, who seeks to benefit, and the problem or issue it seeks to address.

§ Explain the expected results map or results framework, implementation strategies, and the key assumptions underlying the strategy.

§ Link the intervention to WV Somalia national strategy and child well-being targets

§ Identify any significant changes (plans, strategies, logical frame-works) that have occurred over time and explain the implications of those changes for the evaluation

§ Identify and describe the key partners involved in the implementation and their roles.

§ Describe the scale of the intervention, such as the number of components (e.g., phases of a project) and the size of the target population for each component.

§ Indicate the total resources, including human resources and budgets.

§ Describe the context of the social, political, economic, and institutional factors, and the geographical landscape within which the intervention operates and explain the effects (challenges and opportunities) those factors present for its implementation and outcomes.

§ Point out design weaknesses (e.g., intervention logic) or other implementation constraints (e.g., resource limitations).

Evaluation Scope and Objectives

Provide a clear explanation of the evaluation’s scope, primary objectives and main questions.

§ Evaluation scope-define the parameters of the evaluation, for example, the time period, the segments of the target population included, the geographic area included, and which components, outputs or outcomes were and were not assessed.

• Evaluation objectives-spell out the types of decisions evaluation users will make, the issues they will need to consider in making those decisions, and what the evaluation will need to achieve to contribute to those decisions.

§ Evaluation criteria-define the evaluation criteria or performance standards used. The report should explain the rationale for selecting the particular criteria used in the evaluation.

§ Evaluation questions-evaluation questions define the information that the evaluation will generate. The report should detail the main evaluation questions addressed by the evaluation and explain how the answers to these questions address the information needs of users.

Evaluation Approach and Methods

The evaluation report should describe in detail the selected methodological approaches, methods and analysis; the rationale for their selection; and how, within the constraints of time and money, the approaches and methods employed yielded data that helped answer the evaluation questions and achieved the evaluation purposes. The description should help the report users judge the merits of the methods used in the evaluation and the credibility of the findings, conclusions and recommendations. The description of methodology should include discussion of each of the following:

§ Data sources-sources of information (documents reviewed and stakeholders), the rationale for their selection and how the information obtained addressed the evaluation questions.

§ Sample and sampling frame-the sample size and characteristics; the sample selection criteria, the process for selecting the sample (e.g., random, purposive); and the extent to which the sample is representative of the entire target population, including discussion of the limitations of the sample for generalizing results.

• Data collection procedures and instruments-methods or procedures used to collect data, including discussion of data collection instruments (e.g., interview protocols), their appropriateness for the data source and evidence of their reliability and validity.
• *Performance standards***-**standard or measure that will be used to evaluate performance relative to the evaluation questions (e.g., national or regional indicators, rating scales).
• Stakeholder engagement-stakeholders’ engagement in the evaluation and how the level of involvement contributed to the credibility of the evaluation and the results.
• Background information on evaluators-the composition of the evaluation team, the background and skills of team members and the appropriateness of the technical skill mix, gender balance and geographical representation for the evaluation.
• Major limitations of the methodology-major limitations of the methodology should be identified and openly discussed as to their implications for evaluation, as well as steps taken to mitigate those limitations.
• Data analysis-procedures used to analyse the data collected to answer the evaluation questions. It should detail the various steps and stages of analysis that were carried out, including the steps to confirm the accuracy of data and the results. Data from the end line evaluation should be systematically compared with data from the baseline evaluation, and the difference between baseline and end-line values should be tested for statistical significance and discussed. The report also should consider the appropriateness of the analysis to the evaluation questions. Potential weaknesses in the data analysis and gaps or limitations of the data should be addressed, including their possible influence on the way findings may be interpreted and conclusions drawn.

Findings and Conclusions

Present the evaluation findings based on the analysis and conclusions drawn from the findings.

Findings-presented as statements of fact that are based on analysis of the data. The evaluation findings should be structured around the key evaluation questions and project indicators so that report users can readily make the connection between what was asked and what was found. Variances between planned and actual results should be explained, as well as factors affecting the achievement of intended results. The assumptions or risks in the project design that subsequently affected implementation should also be discussed.

Conclusions-this section should be comprehensive and balanced and highlight the strengths, weaknesses and outcomes of the intervention. The conclusion section should be well substantiated by the evidence and logically connected to the evaluation findings. It should respond to key evaluation questions and provide insights into the identification of and/or solutions to important problems or issues pertinent to the decision-making.

Recommendations-the Consultant should provide practical, feasible recommendations directed to the intended users of the report about what actions to take or decisions to make. The recommendations should be specifically supported by the evidence and linked to the findings and conclusions around key questions addressed by the evaluation. This should address the sustainability of the initiative and comment on the adequacy of the project exit strategy.

Lessons Learned:** The report should include discussion of lessons learned from the evaluation, that is; new knowledge gained from the particular circumstances (intervention, context outcomes, even about the evaluation methods) that apply to a similar context. Lessons should be concise and based on specific evidence presented in the report.

Report Annexes

Annexes shall include the following to provide the report user with supplemental background and methodological details that enhance the credibility of the report:

§ ToR for the evaluation

§ Additional methodology-related documentation, such as the evaluation matrix and data collection instruments (questionnaires, interview guides, observation protocols, etc.) as appropriate

§ List of individuals or groups interviewed or consulted and sites visited

§ List of supporting documents reviewed

§ Project results map or results framework

§ Summary tables of findings, such as tables displaying progress towards outputs, targets, and goals relative to established indicators.

11. Anticipated Limitations

Time and security may be a major limitation with regard to assessment processes in fragile and versatile contexts such as Somalia and this makes it often challenging to keep up strictly with a set agenda. In addition, In Somalia; households spent a better part of the afternoon hours in prayers and it will be hard for the enumerators to administer many questionnaires per day (in an effort to complete the assessment timely). To address this issue, firstly WV Somalia will allocate extra overflow days for field data collection. WV Somalia team will also work closely with the security department to ensure that the evaluation field processes are conducted in the most appropriate time and secure conditions. Therefore, the Consultant should be able to demonstrate some level of flexibility when required.

12. Budget

The evaluation is funded through the project budget under logframe code 5.3.1, which has already been approved. The budget will cover all associated costs of the evaluation, including the consultants’ fee, travel and accommodation, field transport and evaluation supplies.

13. Standards of Ethics and Child Protection

The lead evaluator will be responsible for ensuring that data collection and analysis approaches are designed to mitigate child protection risks and protect participants’ privacy and wellbeing by establishing and following credible ethical evaluation principles. The lead evaluator must ensure all members of the evaluation team has been oriented in the ethical considerations employed in the evaluation. Ethical principles will include the following:

Voluntarism, confidentiality and anonymity of participants: All participation in interviews must be voluntary, will not create harm to participants during or after the data gathering, and their anonymity and confidentiality will be protected. Voluntary involvement must be assured by a scripted verbal explanation of the survey being conducted. The script must inform respondents that they may choose to not respond to certain questions and may end the survey at any time.

Do No Harm: Project and evaluation themes must be screened for topics and questions that may cause distress to some interviewees. Mitigating approaches and referral options must be developed accordingly.

Integrity: Data from participants must be presented honestly and proportionately, such as the authoritativeness, extent-shared and intensity of opinions across the target population, and aligning quotes with the evaluative themes intended by the informant. Unexpected or contentious findings should be triangulated with other forms of data to gauge significance.

Participant perspective: To the extent possible, given logistical limitations of each context, preliminary findings should be shared with a plenary of project stakeholders to invite their reactions and interpretations. These will be recorded and added to the final report.

Child Protection: If children (under the age of 18) are to be interviewed, it will be in the presence of a responsible adult from the child’s family, or other implied guardian from the community. Children will not be exposed to questions of a highly personal, sensitive, potentially distressing or embarrassing nature.

If children are to be interviewed, child protection reporting protocols will be established and all staff made aware of when and how to report any issues that arise from data collection.

Evaluation coordinators must have completed and been cleared by a police check within the last two years. All evaluation coordinators and collectors will be required to review, sign, and adhere to a child protection code of conduct.

The lead evaluation must familiarise him or herself with the following ethical and protection guides (to be supplied to the selected lead evaluator):

· WVI Child Protection Code of Conduct

· DFAT Guidelines for Child Protection

· DFAT Prevention of Sexual Exploitation, Abuse and Harassment Policy

· WVI Guideline of Ethical Principles

· Australasian Evaluation Society Guidelines of Ethical Principals

· BOND Tool for Evidence Principles

14. Documents to be made available for evaluation preparation

· WVA Templates for Evaluation Plan and Evaluation Report

· Project design documents: narrative, logframe and M&E Plan

· Project monitoring reports, including annual progress reports and Indicator tracking table

· Project baseline study (and data if applicable)

· Any previous external reviews of the project

· Training reports

· Success stories

· Any district-level secondary data and other relevant documents and reports

· BOND Evidence Principles Checklist

· Australasian Evaluation Society Ethics Guidelines