TERMS OF REFERENCE (TOR)

INDEPENDENT MID-TERM EVALUATION OF THE PROJECT:

“YOUTH ENVIRONMENTAL IMPACT (YENI)”

BACKGROUND INFORMATION

Caritas Switzerland (CACH) is an independent Swiss NGO, engaged in Switzerland and more than 20 countries worldwide, including Kosovo. Our mission is to prevent, combat and eradicate poverty in a sustainable manner. To achieve this goal, CACH employs a policy of equity: we support inclusive communities with opportunities for everyone regardless of their ethnic background, group affiliation, or gender. CACH has been working in Kosovo since 1999, focusing on social inclusion, rural development and rehabilitation. Based on the Strategy 2021-2025, our central themes are Climate Change, Income Generation and Migration.

Overall, the CACH Country Programme for Kosovo is based on CACH’s International Cooperation Strategy and it defines country-specific outcomes. Within the theme of Climate, CACH in Kosovo is committed to climate justice and to promoting the inclusion of women and members of minority groups.

The Western Balkans, including Kosovo, is one of the regions in Europe that is most heavily affected by the impact of climate change. This trend is projected to continue, with estimates of temperature increases of 1.7 - 4.0°C and even exceeding 5.0°C by the end of the century, depending on the global effort in the reduction of greenhouse gas emissions. Kosovo is also prone to a wide variety of natural hazards – including floods, landslides, droughts, earthquakes and wildfires – that could pose serious damage to the economy, fiscal balance and well-being of vulnerable populations. Many of these climate-related hazards are expected to magnify with future climate change. These climate and disaster risks can seriously affect productive sectors of the economy such as agriculture, infrastructure, energy, water resources, communities and households. Kosovo’s agriculture sector makes up 11 percent of the Gross Domestic Product, but it has exhibited low growth and has been affected by weather conditions and climate change.

Kosovo’s Constitution of 2008 recognizes the protection of the environment as one of the main values (art. 7) and “nature, biodiversity, environment and national inheritance as everyone’s responsibility” (art. 14). The Government of Kosovo has adopted the energy strategy, which will have a huge impact on Kosovo’s economy. A Climate Change Strategy is in place and the climate change law was finalized and is awaiting adoption. If fully implemented, this could provide a sound basis for tackling climate change.

The YENI project goal is to improve climate action and environmental protection in Kosovar municipalities through the empowerment and active engagement of youth. By putting a long-term emphasis on youth via schools and climate education in the local context, this project aims to support Kosovar local authorities in enhancing their ability to address the country's climate change vulnerabilities. To contribute to this overall goal, the project activities are focusing on tackling existing barriers to make behavioural change constructively possible. Therefore, the project will facilitate the strengthening of the role of youth as agents of climate change in the Kosovo municipalities.

The YENI project aims to motivate youth for climate change engagement and vision, as well as complementing skills, enabling them to become active citizens and contribute to climate change mitigation or adaptation. Therefore, the project will facilitate the strengthening of the role of youth as agents of climate change in the targeted municipalities.

YENI project is in line with the Sustainable Development Goals (SDGs), specifically, SDG 4 (Quality Education) and SDG 13 (Climate Action), which emphasize the special importance of environmental education and awareness-raising on topics related to climate change and its associated impacts. Education is a crucial component of climate change mitigation and adaptation. Increasing environmental awareness and education on the climate crisis can contribute to the effective reduction of greenhouse gas (GHG) emissions and to adapting to a changing climate in both the short and long term.

1. PURPOSE OF THE EVALUATION

The mid-term review should explore YENI’s work and achievements and support to assess progress so far and provide guidance on how to maximize the potential for achieving the intended results and improve learning in its remaining timeframe (2023-2024). Through the assessment of the performance, achievements and lessons learned to date, the review will contribute to both - learning and accountability. This evaluation will adhere to the OECD/DAC[1] and the SEVAL[2] standards for evaluation. For all practical purposes, this TOR defines the overall scope of this evaluation.

The main purpose of this mid-term independent evaluation is to provide an independent assessment of the progress to date, through an analysis of the relevance, effectiveness efficiency and impact of the project. The evaluation should give furthermore a brief outlook on the potential future impact of the project.

The specific objectives of the mid-term evaluation are the following:

1. Assess the relevance of the project’s design, regarding country needs and how the project is perceived and valued by the target groups.
2. Evaluate the relevance of the YENI project to address the environmental and climate change issues for youth in the 17 targeted Kosovar municipalities. It will also assess the relevance of the stakeholders targeted by the intervention and the methodologies and approaches to do so.
3. Identify the contributions of the project to the Sustainable Development Goals 13 climate action and SDG 4 Education and its synergy with the overall CACH objectives and strategy.
4. Assess the effectiveness of the YENI project at achieving its objectives and provide clear insights about what has and has not worked so far and why, including unexpected results and factors affecting project implementation (positively and negatively).
5. Assess the efficiency in terms of value for money of the delivery of the YENI outputs.
6. Briefly assess and give a general outlook on the potential impact of the project.
7. Identify lessons and provide a set of actionable recommendations on how the project and the project coordination/management could be adjusted for further improvement and to strengthen the delivery of results.
8. Provide strategic recommendations for the different key stakeholders to improve the implementation of the project activities and attainment of project objectives.

Recommendations emerging from the evaluation should be strongly linked to the findings of the evaluation and should provide clear guidance to stakeholders on how they can address them. It is expected that the findings and recommendations of this mid-term review will help to identify any needed course corrections in the project’s approach and activities and bring valuable external reflections to help strengthen the project and complement the MEAL system of the project through an adaptive management modality.

1. BRIEF DESCRIPTION OF THE PROJECT INTERVENTION

The YENI project is designed to help the Kosovar local authorities to increase their capacity to address the country’s climate change vulnerabilities by focusing long-term on youth via schools and education in the local context. To accomplish this overall goal, the project activities are focusing on tackling existing barriers to make behavioural change constructively possible. Therefore, the project will facilitate the strengthening of the role of youth as agents of climate change in the municipalities.

The overall objective of the project is:

• Improved climate action and environmental protection in Kosovar municipalities through mobilization and active engagement of youth.

This shall be accomplished by educating and mobilizing youth from 60 high schools across Kosovo on climate change and environmental issues. The topic will be strengthened by establishing and strengthening Eco-Clubs and utilizing existing student councils in the field of climate and environmental protection. As a result, mobilized youth shall actively participate in developing Local Environmental Action Plans (LEAPs) within their municipalities.

The three outcomes of the project are as follows:

• Youth of secondary schools is actively undertaking sustainable measures and is recognized as a partner for climate action by the local and national authorities.
• Improved climate action and environmental protection in Kosovar municipalities through active engagement of youth.
• Municipal Education Directorates, Student Councils, Eco-Clubs and pupils succeed with their Climate Change initiatives in their municipalities.

CACH, respectively the YENI project, aims to provide youth with a climate engagement and vision, as well as complementing skills, to enable them to become active citizens and make their contribution to mitigating or adapting to climate change (CC). Eco-Clubs or similar youth mechanisms are not established in all upper secondary schools in Kosovo. However, most schools have established Student Councils, nevertheless, only a small part deals with environmental protection. Therefore, with climate change education and environmental skills development young Kosovar girls and boys can play an active role in protecting and improving the environment. They can change their lifestyle and how it affects the environment. Moreover, they can build social and individual capacities and attitudes toward climate change. They can make their homes, schools and youth organizations more environmentally friendly by adopting climate-friendly practices, recycling different materials as well as preserving resources such as water and electricity. Engaging youth in environmental protection not only directly impacts changing youth behaviour and attitudes but possibly influences their parents, relatives and families.

Eco-clubs in schools encourage students to engage in meaningful environmental activities and projects. It is a forum for students to reach out and influence their parents and neighborhood communities to promote responsible environmental and climate-friendly behavior, enabling students to go beyond the confines of a syllabus or curriculum to investigate environmental concepts and actions. While everyone, everywhere emphasizes the importance of “learning to live sustainably”, the environment and concrete behavior change remains a secondary concern in the formal education system. Creating awareness and sensitivity among students toward environmental issues and their related problems is only a start for that. Imparting knowledge and nudge on behavior change helps individuals and social groups gain a variety of experiences and acquire a basic understanding of the environment and its associated problems, but also helps to adapt the behavior of the next generation.

Building attitudes to help individuals and social groups acquire a set of values and feelings of concern for the environment and the motivation for actively participating in environmental protection. Eco-Clubs are one way to encourage interest and involvement and can be extremely rewarding, especially when students are given the responsibility to make their own decisions about the changes they want to see and the opportunity to improve their school environment.

An additional outcome envisaged is reserved for the inclusion and contribution of school youth to the development of the LEAPs, in which the suggestions of the youth shall be incorporated. Furthermore, this document must be harmonized with other local-level documents to create synergies of actions with other actors that are active at the municipal level. Lastly, in this process, the Student Councils and Eco-Clubs shall harmonize their efforts with other youth and environmental NGOs and CSO groups to increase advocacy and lobbying efforts and to have a bigger impact on local authorities. Advocacy and lobbying are essential so that youth is not left behind in the conversation, especially young people in the upper secondary schools in Kosovo which are the key actors in the sustainable development process. Youth advocacy is a key factor in the development and communication of climate change issues and environmental protection.

1. SCOPE & FOCUS OF EVALUATION / EVALUATIVE QUESTIONS
   1. SCOPE OF THE EVALUATION

The mid-term evaluation will analyze all planned outputs and outcomes under the project within this period (please see Table 1), with particular attention to synergies between the components and contribution to climate change and environmental issues and youth in Kosovo.

The evaluation will discuss, throughout the evaluation questions, how the project is addressing its main issue (i.e., climate change) and the CACH cross-cutting themes including conflict sensitivity, gender, partner support and participation. The evaluation should help to understand how and why the project has obtained or not the specific results from output to potential impacts, how changes in the context have influenced progress, and what adjustments are needed.

Furthermore, the evaluation process will assess the first implementation phase (18 months) of the YENI project activities conducted and implemented during the period 2022 – 2023 to provide improvements based on the findings from the field and support further project implementation recommendations/phase.

The evaluation experts/service provider is not allowed to share any information and/or evaluation results without prior notification to the CACH/YENI project staff. The service provider will sign a copy of the Code of Conduct document and attach it as Annex to the service agreement.

Table 1: Scope & focus of the evaluation.

At the central level, the co-funder Ministry of Regional Development, the University of Prishtina and the Ministry of Environment, Spatial Planning and Infrastructure, Ministry of Education, Science, Education and Innovation.

Regionally the following target municipalities are part of the YENI project:

Table 2: Geographical coverage

1. EVALUATIVE QUESTIONS & FOCUS

Concerning the four DAC criteria (Relevance, Effectiveness, Efficiency, Impact), the following questions are of special interest:

1. Relevance

• How appropriate and relevant is the YENI project towards the alignment with the national and international policies on environmental and climate as: National Climate Strategy National, Youth Strategy, Agenda 2030 and Sustainable Development Goals (4 – Education, 7 Affordable and Clean Energy, and 13 Climate Action) as well as the Local Environmental Action Plans (LEAPs) of 17 target municipalities.
• How consistent are the activities and outputs with the intended effects?
• Is the project relevant to the identified needs of beneficiaries (target group of project)?
• What is CACH/YENI project's position towards cooperation with relevant stakeholders/actors in climate education and youth?
• To what extent is the project fit for purpose to:

1. Encouraging young people to take sustainable action on climate change issues by helping local and national authorities acknowledge them as partners in the fight against climate change.
2. Increase youth knowledge and engagement in environmental protection and climate action in Kosovar municipalities.
3. Developing awareness among secondary school students regarding the field of climate change and environmental protection.
4. The development of new climate change materials for the high school level. Ensuring that the students, student councils, eco-clubs, and MEDs in each municipality are successful in their climate change projects.

• Has there been any change since the project was formulated that might have affected its relevance? If so, what are these changes and to what extent has the project managed to adapt to ensure it remains relevant?

1. Effectiveness

• To what extent has and will the YENI project deliver on its outputs and outcomes at Kosovo regional level to date? Were there any positive unintended consequences or unexpected effects? In particular:

1. How effective has YENI been so far in developing methodologies to raise awareness on climate change and environmental protection matters? How effective has YENI been in collecting the data that feed into these methodologies?
2. How effective has YENI been in engaging with key decision makers in a country to mainstream climate change in policy and decision making? What are the early markers of changes among decision/policy makers that demonstrate YENI is on its way to tackle existing barriers to make behavioral change constructively possible and strengthen of the role of youth as agents of climate change in the municipalities?

• For all the above questions, what are the factors influencing positively and negatively the effectiveness of the project?

1. Efficiency

• To what extent are the YENI outputs in balance with the level of effort, time and resources spent?
• To what extent spending and project delivery progress according to the planned schedule?
• How much has the project management adapted to any changing condition to improve project implementation efficiency?
• To what extent has the project built on existing agreements, initiatives, data sources, synergies and complementarities with other projects, partnerships, etc. and avoided duplication of similar activities by other groups and initiatives?
• Which alternative approaches could have led to equivalent results at lower costs?

3.2.4 Impact

• To what extent does the project have the potential to have made a serious contribution to the overall goal of the project by the project's end?

“Improved climate action and environmental protection in Kosovar municipalities through mobilization and active engagement of youth.”

The evaluators are requested to give a general and forward-looking assessment of the Impact question only.

1. EVALUATION METHODS AND PROCESS

The independent mid-term evaluation will comply with the OECD/DAC and the SEVAL standards for evaluation. The evaluation will apply a mixed methods approach, engaging with key stakeholders of the project at all levels during the design, fieldwork, validation and reporting stages.

The evaluator(s) is expected to develop an evaluation framework based on the suggested key evaluation questions above but may suggest additional questions or modifications. The inception report will be prepared as the first deliverable of the evaluation and will include an evaluation matrix[3] presenting how the key issues will be addressed, the data sources and the data collection methods that will be used for the evaluation and a set of criteria to rate the strength of the evidence collected.

All data collection tools are to be included as annexes to the final evaluation report. The link between evaluation questions, data collection, analysis, findings and conclusions must be made and set out transparently in the presentation of the evaluation findings. Conclusion and recommendations should be underpinned by a strong set of evidence.

The evaluation will seek the views of the range of stakeholders who have been engaged in the process to date[4] to conclude whether the project is on track and expected to realize its set objectives.

To collect the data for analysis, the evaluation will make use of the techniques listed below (but not limited to). The data from these sources will be triangulated[5] to increase the validity and rigour of the evaluation findings.

• Review of relevant documentation from the project (documents to be shared once the evaluator has been selected).
• Desk review: project design and strategy documents, activity documents, communications, research and publications.
• Key informant interviews and focus groups: with project staff, tripartite constituents, beneficiaries of the project and potentially other stakeholders and partners.
• Field in-depth interviews: the evaluation team is expected to meet project beneficiaries’ men and women to undertake more in-depth reviews of the project work and results.

The evaluator must indicate the selection criteria for individuals to interview.

The selection of the field visit locations should be based on criteria to be defined by the evaluator.

• Presentation of the preliminary findings to the key stakeholders in a workshop.

At the end of the fieldwork the evaluation team will present preliminary findings to the project's key stakeholders in a workshop to discuss and refine the findings and fill information gaps.

The methodology should include examining the interventions, specifically in the light of the project’s log frame, logical connection between levels of results, its coherence with external factors, their alignment with the CACH’s strategic objectives, and Agenda 2030 – SDG(13 - Climate action and 4 - Education) and related targets.

The data and information should be collected, presented and analyzed with appropriate gender, ethnicity and where feasible in terms of disability disaggregation, even if the project design did not take those into account. A more detailed methodology for the assignment will be elaborated by the evaluator based on this TOR, in consultation with the CACH/YENI team and key stakeholders in the inception report.

1. DELIVERABLES

5.1. An inception report (not more than 20 pages excluding the annexes) – upon the review of available documents and an initial discussion with the project management, the inception report will be developed, including:

• Conceptual framework that will be used to undertake the evaluation.
• Own methodology proposed.
• Data required to answer the evaluation questions, data sources by specific evaluation questions (emphasizing triangulation as much as possible) data collection methods, and sampling.
• Selection criteria for locations to be visited.
• Selection criteria for individuals for interviews (as much as possible should include men and women).
• Detail the work plan for the evaluation, indicating the phases in the evaluation, their key deliverables and milestones.
• Set out the list of key stakeholders to be interviewed and the tools to be used for interviews and discussions.
• Set out the agenda for the stakeholder’s workshop.
• Set out an outline for the final evaluation report.
• Interview and focus group guides.
• Evaluation question matrix - attached as Annex III to this TOR.
  1. Preliminary Findings to be shared with the key stakeholders – After completion of data collection or before sharing the draft report, the evaluator should present preliminary debriefing and findings to the key stakeholders. The evaluator will set the agenda for the half-day to one-day meeting. The presentation should provide a brief review of key results for each evaluation criterion. The workshop will be technically organized by the evaluation team with the logistic support of the YENI project.

The inception report should be approved by the Project Manager and the MRD coordinatorbefore proceeding with the fieldwork. More detailed instructions can be found in Annex II - Writing the inception report.

1. First draft of Evaluation Report – The report should be no longer than 30 pages excluding annexes. The Project Manager, Country Director and YENI coordinator from the MRD will review the first draft of the Mid-Term Review (MTR) report, to ensure that it meets the required quality standards and covers all agreed components and contents of the MTR. Detailed comments and feedback on the draft report will be provided to evaluation consultant(s) and discussion may be held to provide clarifications as necessary. The draft review report will also be shared with all relevant stakeholders for additional feedback and input. The evaluator(s) should submit a comprehensive draft report consisting of major findings and recommendations for future courses of action.
2. The final version of the evaluation report will be produced by the evaluation consultant(s) based on feedback received on the draft report, incorporating comments received from CACH/YENI, the MRD and other key stakeholders. The report should be no longer than 30 pages excluding annexes and executive summary. The final draft report should be submitted within the given timeline with enough detail and quality.

The draft and final versions of the evaluation report in English (maximum 30 pages plus annexes) will be developed under the following structure (Annex IV - Preparing the evaluation report).

Table 3: Structure of report

1. SCHEDULE

This assignment is anticipated to take place between 10.08.2023 - 10.10.2023. The total duration of the evaluation process is estimated to be 30 working days for both national evaluators/experts (each 15 days).

Table 4: Evaluation Timetable and Schedule

Table 5: Proposed workdays for the evaluation team

1. EVALUATION TEAM / QUALIFICATIONS

The evaluation team should consist of two national evaluation specialists/consultants, who will work jointly to achieve the expected results. Both national evaluation specialists will work in close coordination with the YENI Project Manager and MRD coordinator. The project team will provide administrative and logistical support as needed.

Evaluation Manager: the evaluation will be accompanied by Mr Bardh Xërxa - YENI Project Manager, who is responsible for completing the following specific tasks:

• Brief the evaluator on CACH evaluation policies, procedures and sharing of relevant documents for desk review.
• Initial coordination with the project team on the development of the field mission schedule and the preliminary results workshop. and feedback to the inception report.
• Circulate the first draft of the evaluation report for comments by key stakeholders and CACH to report draft.
• Ensure the final version of the evaluation report addresses stakeholders’ comments (or an explanation why any has not been addressed) and meets CACH requirements, approvals and payment.

The National Evaluators should have the following qualifications:

• Education:
• Master’s degree in one of the following relevant fields: Environmental Economics, Education, Development Studies, Public Finance, Public Management, Management or other related field.
• Academic/professional background in M&E, RBM or programme management is an asset.
• Work Experience:
• At least 5 years overall professional experience as an evaluator.
• At least 5 years of relevant professional work experience in climate change, environmental management, youth, education, or performance/results-based budgeting.
• Track record in conducting project evaluations.
• Experience working with international development partners, especially in climate change, youth, environmental management or public financial management.
• Experience in working with OECD/DAC or the SEVAL standards.
• Proven experience with logical framework approaches and other strategic planning approaches, M&E methods and approaches, information analysis and report writing.
• Extensive knowledge of, and experience in applying, qualitative and quantitative research methodologies.
• Functional competencies:
• Strong interpersonal and communication skills.
• Strong analytical, reporting and writing abilities.
• Excellent speaking and presentation skills.
• Openness to change and ability to receive and integrate feedback.
• Language requirements:
• Excellent spoken and written Albanian and English language skills required. Other local languages, in particular Serbian, are considered an asset.

1. BUDGET

This assignment foresees a total of 30 days for the two (2) evaluators, to cover the whole evaluation process.

The available budget foreseen for the evaluation process is EUR 7,000.00.

The following costs will be covered by evaluators (i.e., they need to be included in the offer, in addition to daily fees of each evaluator):

• Transportation cost in Kosovo.
• Half a day workshop and necessary translation costs.

CACH/YENI project will not pay any additional costs/expenses that may arise from the evaluation process except the financial amount presented and agreed on in the service agreement with the evaluator(s).

ANNEXES

• Annex I: Bidding Instructions.
• Annex II: Writing the inception report.
• Annex III: Evaluation question matrix (EQM).
• Annex IV: Preparing the evaluation report.
• Annex V: General conditions for procurement.
• Annex VI: Exclusion criteria.