End of Project Evaluation

Terms of Reference End of Project Evaluation

Project Title: Vision for Zambia July 2019

• Project Code 2306
• Project Title Vision for Zambia
• Cost Centre 74
• Country Zambia
• Cost Centre Director Lucia Nadaf
• Project Officer Fabian Miselo
• Partner(s) Ministry of Health, Provincial and District Health Offices
• Kitwe Teaching Hospital-Eye Annexe
• Start and End Date 1 July 2016 to 30 June 2019

1.BACKGROUND:

Project Overview

In 2011, it was estimated that around 800 children had untreated cataract on in the Copperbelt Province of Zambia. Each year between 150 and 190 children were born with or develop cataracts and many more require treatment for other causes of blindness, such as trachoma, trauma and glaucoma, or simply required spectacles. Child eye health human resources had been particularly poor, with specialised paediatric services virtually non-existent before Orbis’s initial intervention in 2010. In response to the situation described, Orbis began its investment in Zambia at the tertiary-level hospital Kitwe Eye Annex (KEA) in 2010 and over a period of 3 years developed human resources and provided much-needed equipment and consumables, improving service delivery for paediatric eye health. As a result of these interventions, KEA was now able to accept increasing numbers of cases.
However, challenges still existed:

• Many children were not accessing public healthcare services due to travel costs,
  travel distance, and cultural beliefs around biomedicine.
• Due to a lack of training and resources at primary and districts levels, children who
  managed to visit a public health facility were often misdiagnosed. These cases were
  then not treated or referred appropriately. Mistrust existed in and about the public
  healthcare system as a result.
• Many children presented directly at KEA for simple eye problems such as
  conjunctivitis, an infection that can be treated with eye drops dispensed at a primary
  or district level.

Therefore, the vision for Zambia project which began in July 2016 with 3 years funding from DFID intended to extend support to all the 10 districts of the Copperbelt province by building the capacity of key healthcare personnel at primary, secondary and tertiary levels, and by strengthening referral systems, to ensure high quality eye healthcare services are delivered.

Project Goal

The project goal is to reduce childhood blindness and visual impairment of targeted children within the 10 districts of the Copperbelt province in the Zambia.

Expected Outcome

89,945 children (0-14 years) on the Copperbelt Province take up quality eye health services and follow up care at the appropriate level.

Expected Outputs

Output 1: Increased community awareness of eye health conditions and services available through targeted communication campaigns including facilitated film screenings, radio shows and distribution of IEC materials in 10 districts of the Copperbelt Province.

Output 2: 766 Health Professionals at community, primary and district level, in 10 districts in Copperbelt province have improved capacity to identify, screen, treat basic conditions where appropriate and refer children with eye health conditions.

Output 3: 11 Health Personnel (Paediatric ophthalmologists, nurses and biomedical staff) at Kitwe Eye Annex have increased capacity to provide high quality sub speciality paediatric eye surgery and post op care through hospital based programmes

Output 4: Improved coordination between district health personnel in the Copperbelt Province in planning, service delivery, feedback and information sharing of referrals.

To achieve the above, the following interventions funded under this grant include:

• Facilitated film screenings within the community.
• Radio messaging
• Training of health professionals at community, primary and district level to identify,
  refer and treat where appropriate, children with eye health conditions.
• Equipping health facilities with small eye health diagnostic equipment
• Training of health professionals at tertiary level on skills to conduct paediatric eye surgery and treatments during hospital- based programmes.
• Facilitation of co-ordinated meetings amongst district health personnel.

2. PURPOSE, AIM, OBJECTIVES AND AUDIENCE FOR EVALUATION

Aim and Purpose

In addition to Orbis International Zambia’s commitment to capture learning, the organisation is also contractually obliged to conduct a final evaluation on programme activities funded by the UK Government grant. In order to comply with this, Orbis International Zambia is commissioning an evaluation that focuses on results achieved, efficiency and effectiveness of implementation as well as quality of administration. Therefore, the purpose of the final evaluation is to ascertain to what extent the interventions applied in this project resulted in the planned outcomes and outputs. These are:

Outcome: 89,948 children in the Zambia Copperbelt Province take up quality eye health
services and follow up care at the appropriate level.

Output 1: Increased community awareness of eye health conditions and services available through targeted communication campaigns including facilitated film screenings, radio shows and distribution of IEC materials in 10 districts of the Copperbelt Province.

Output 2: 766 Health Professionals at community, primary and district level, in 10 districts in Copperbelt province have improved capacity to identify, screen, treat basic conditions where appropriate and refer children with eye health conditions.

Output 3: 11 Health Personnel (Paediatric ophthalmologists, nurses and biomedical staff) at Kitwe Eye Annex have increased capacity to provide high quality sub speciality paediatric eye surgery and post op care through hospital- based programmes

Output 4: Improved coordination between district health personnel in the Copperbelt Province in planning, service delivery, feedback and information sharing of referrals. Audience

The main audience for the final evaluation report are not limited to Orbis, Mannion Daniels, DFID, and the Ministry of Health.

3. KEY LEARNING AREAS AND QUESTIONS

Relevance: To assess the extent to which the programme design met the identified needs

• Was the project relevant to the identified needs of the target beneficiaries and the context?
• Were the project inputs and execution strategy adequate to achieve the project goal, outcome and outputs?
• To what extent do achieved project goal, outcomes and outputs continue to be relevant to the needs of Copperbelt District?

Effectiveness: To assess the quality of the intervention’s work by assessing the rate at which outputs are converted into outcomes and impacts, and the cost-effectiveness of this conversion.

• To what extent did the project reach the targeted beneficiaries at the project goal and
  outcome levels? How many beneficiaries have been reached?
• To what extent were the intended project goal, outcomes and outputs achieved and
  how?
• How effective were the strategies and tools used in the implementation of the project?
• How appropriate were the management processes in supporting delivery of project
  outputs?
• To what extent did the beneficiaries and identified stakeholders participate in the
  planning and implementation of the project?
• To what extent did the project approach to Monitoring, Evaluation and Learning (MEL) contribute in meeting project results?
• To what extent has this project generated positive (or negative) changes in children (0-14 years) on the Copperbelt Province accessing quality eye health services and follow up care at the appropriate level.

Efficiency: To assess how efficiently the project is delivering its outputs, considering the rate at which intervention inputs are converted to outputs and its cost-efficiency.

• Have the results achieved justified the costs incurred? Have the resources been effectively utilised?
• Has the project been implemented and managed as planned in the project proposal logframe and budget?
• How well did the partnership and management arrangements work and how did they
  develop over time?
• How well did the financial systems work?
• Did identified risks occur? How well were risks managed?
• Were the processes in place to ensure value for money effective?

Equity: degree to which the results of the intervention are equitably distributed.

• To what extent did the project reach the most vulnerable and marginalised children?
• To what extent did the project address gender equality?

Impact: To assess the potential impact of the project on the targeted communities

• What was the programme’s overall impact and how does this compare with what was expected?
• What are the significant changes consequences (positive and negative/intended and
  unintended) that are a result of the project intervention?
• What difference has been made to the lives of the target beneficiaries?

Sustainability: To assess the potential for the continuation of the impact achieved and
of the delivery mechanisms, following the withdrawal of external support.

• Are there any plans and/ or strategies to sustain the gains made in this project?
• Was the exit strategy to phase out UK Aid funding effective?
• What key areas will require additional support in order to improve prospects of sustainability of the project achievements and the potential for replication of this approach?

Lessons and Recommendations: To identify key lessons that can be used to guide future strategies projects and broader development lessons.

• What are some of the key lesson learned at a result of this project that can be shared
  and replicated
• What are the recommendations for improvements (best practice) based on
  observations during the evaluation process (e.g. for sustainability, future programme
  design and management).

4.EVALUATION SCOPE

The evaluation will cover the entire project duration, from July 2016 to June 2019 and is an indepth analysis of all result areas, beneficiaries, partners and project stakeholders. It will seek to assess the impact and relevance of the project’s interventions in achieving the impact. It will also examine project management and activity implementation with a view to addressing the results achieved, the partnerships established, as well as issues of organisational capacity.

5. KEY DELIVERABLES

• Evaluation Inception report with detailed workplan, methodology and tools.
• A presentation of provisional evaluation findings/ first draft for face-to-face discussion between the Consultant and Orbis International Zambia
• Final evaluation report, not exceeding 30 pages (excluding annexes), of publishable
  quality, including an executive summary and with consistency between findings,
  conclusions, lessons and recommendations, approved by Orbis International Zambia
• Three case studies highlighting project outcomes (both positive and negative)
• Appendices, including a list of informants and the evaluation team’s work schedule
  (MSWord), electronic copies of survey findings, notes from meetings, key informant
  interviews and focus group discussions.

6. SUGGESTED EVALUATION REPORT OUTLINE

The following is a tentative outline for the final evaluation report (maximum 30 pages,
excluding annexes)

A. Executive summary highlighting main findings and recommendations.
B. Introduction
C. Project background
D. Evaluation Objective (s)
E. Evaluation methodology including limitations
F. Results
G. Lessons learned (both positive and negative) and/or good practice
H. Recommendations
I.Conclusion
J. Annexes: All evaluation tools, evaluation schedule, and other supporting documents
(e.g. photos, documentation, and case studies/most significant change stories)

7. SKILLS AND EXPERIENCE

• Orbis invites bids from consultants with excellent knowledge of monitoring and
  evaluation in theory and practice, and a good understanding of the contextual issues
  on the ground. The consultant should have the following qualifications and
  competencies: A master’s degree in monitoring and evaluation, social sciences, Public
  Health or related fields.
• A minimum of five years professional experience in evaluation of internationally funded projects. Experience of DFID evaluations is an added advantage.
• Experience of the logical framework approach, using participatory and mixed methods (qualitative and quantitative) for data collection and analysis in project evaluation.
• Knowledge of health and eye health issues (knowledge of eye health would be advantageous)
• Knowledge and awareness of gender and the experience in the Zambian context Changing the way Zambia sees.
• Ability to write concise and analytical reports and develop relevant, useful recommendations.

8. TEDERS/BIDS

Tenders (technical and financial) should include:

• A cover letter & CV demonstrating how the skills and competencies described above are met, with concrete examples.
• Detailed methodology and tools
• Detailed work plan/timeframe
• An indication of the amount the consultant expects to be paid in fees, and proposed budget.

9. PAYMENT SCHEDULE

• 1st instalment (40%) will be made upon signing of the contract.
• 2nd instalment (30%) will be made upon submission of first draft.
• 3rd instalment (30%) will be made upon submission of final report that meets the standards outlined in this TOR and agreed in the contract with Orbis International Zambia.

10. How to apply:

Interested applicants are requested to submit tenders via email to [email protected] and copy: [email protected] on or before 5:00 pm 7th June 2019. Please use “V4ZEnd of Project EvaluationTechnical and Financial Proposal” in the email subject line.

Shortlisted candidates will be contacted by 12th of June 2019. Shortlisting will be carried out on an ongoing basis.