Terms of Reference (ToR)

Independent Final Evaluation of Let's Debate Change Project in Kosovo

Background Information

Solidar Suisse is a Swiss non-profit organisation which campaigns worldwide for decent work and democratic participation and implements humanitarian action. The NGO has been involved in international cooperation for more than 80 years and builds on long experience and institutional expertise in both development cooperation and humanitarian action. Solidar Suisse was founded by the Swiss Federation of Trade Unions and the Social Democratic Party as Swiss Labour Assistance in 1936. Organized as an association, the bulk of our membership consists of collective members and several thousand single members. Solidar Suisse is a member of SOLIDAR, a European network of more than 60 Civil Society Organisations (CSOs) working to advance social justice in Europe and worldwide. We are a partner of Alliance Sud, the Swiss alliance of some of the largest NGOs active in International Cooperation.

Introduction

The "Let's Debate Change" project, implemented across Kosovo, Serbia, and Bosnia and

Herzegovina, focuses on enhancing youth debate skills, civic participation, and local improvement projects. Solidar Suisse SEE is seeking interested experts or companies to conduct a comprehensive evaluation of this project, assessing its impact, effectiveness, and sustainability to provide actionable insights for improving these projects in the upcoming years.

Purpose of evaluation

The final evaluation should explore the work and achievements and support to assess overall progress. The evaluation will analyze the whole scope of the project and its results framework, which comprises all elements of the intervention logic. Through the assessment of the performance, achievements and lessons learned to date, the review will contribute to both – learning and accountability.

The main purpose of this final independent evaluation is to provide an independent assessment of the overall progress, through an analysis of the project's relevance, effectiveness, efficiency, impact and sustainability. The evaluation should give a brief outlook on the project's potential future impact.

Specific objectives The evaluation aims to:

Relevance:

• Assess the relevance of the project's design concerning country needs and its value as perceived by the target groups.
• Evaluate how effectively the project addresses youth issues in the targeted municipalities and the relevance of the stakeholders involved, including the methodologies and approaches used.

Effectiveness and Impact:

• Assess the effectiveness of the project in achieving its objectives, providing insights into what has worked and what has not, including unexpected results and factors that have positively or negatively impacted project implementation.
• Give a general outlook on the project's realized impact and measure the achievement of its goals across Kosovo, Serbia, and Bosnia and Herzegovina.
• Identify strategies and interventions that contributed to the project's outcomes and analyze the challenges faced during implementation and how they were managed.

Sustainability:

• Assess the sustainability of the project, considering whether its benefits will last beyond its completion. Evaluate the potential for project outputs and benefits to continue and remain sustainable after the project's conclusion.

Lessons and Recommendations:

• Identify lessons learned and provide actionable recommendations for future project phases, focusing on addressing identified shortcomings and enhancing the effectiveness of similar projects in the upcoming years based on evaluation findings.

Recommendations emerging from the final evaluation should be strongly linked to the evaluation findings and should provide clear guidance to stakeholders on how they can address them. It is expected that the findings and recommendations of this final review will help to identify any needed action that was lacking and can be used for the potential future phase of the project.

Scope

The evaluation will cover:

• Effectiveness of youth participation in debate clubs, focusing on local improvement projects and the discussion of issues in local position papers.
• The project's role in building youth's skills in debating, writing and implementing projects and identifying issues of concern – translated into local position papers within the targeted municipalities.
• Impact of local improvement projects and local positions papers.
• Collaboration between municipalities schools, debate club members, and the implementing partners in Bosnia and Hercegovina and Serbia, and its impact on project outcomes.
• Sustainability of the project with special emphasizes on debate clubs, local improvement projects, and local position papers post-implementation.

Methodology

The evaluation will utilize:

• Review of relevant documentation from the project (documents to be shared once the evaluator has been selected).
• Key informant interviews and focus groups: Surveys, interviews, focus groups, and document analysis involving students, school staff, municipal officials, and project partners.
• Desk review: project design and strategy documents, activity documents, communications, research and publications.
• Measurement of key performance indicators (KPIs) related to participation rates, project activities, and outcomes.
• Insights from stakeholders' experiences, including success stories and lessons learned.
• Comparative evaluation of results against project objectives and targets.

Regional evaluation approach

• Kosovo

The assessment will be conducted with physical presence. This includes face-to-face focus group meetings with school directors, quality coordinators, and debate club members, consultations with the project manager and a review of all project documents.

• Serbia and Bosnia and Herzegovina

The assessment will be conducted online/remotely. This includes virtual focus group meetings with school directors, municipal officials, professors involved in the project, and debate club members, as well as virtual meetings with the implementing partners (project manager and/or director).

Target sample

The evaluation will involve:

• The youth participants across 9 debate clubs in Kosovo, 6 debate clubs in Serbia, 6 debate clubs in Bosnia and Herzegovina.
• School directors, quality coordinators, and other school staff involved in the project.
• Community representatives where local improvement projects took place
• Municipal officials of education and youth and sport departments
• Project implementing team

Deliverables

The evaluation will produce:

• Conceptual framework used to undertake the evaluation.
• Own methodology proposed.
• Data required to answer the evaluation questions, data sources by specific evaluation questions, data collection methods, and sampling.
• Detail the work plan for the evaluation, indicating the phases in the evaluation, their key deliverables and milestones.
• Set out the list of key stakeholders to be interviewed and the tools to be used for interviews and discussions.
• Set out an outline for the final evaluation report.
• Interview and focus group guides.
• Evaluation question matrix - attached as Annex III to this TOR.
• First draft of Evaluation Report
• A detailed report with findings, insights, and recommendations.
• A presentation summarizing key outcomes and recommendations for stakeholders.
• Documentation of best practices and lessons learned to guide future project iterations.

Structure of report

Cover page

Table of contents

Acronyms and abbreviations Executive summary

1. Introduction
2. Methodology
3. Project Description
4. Findings
5. Conclusions
6. Recommendations and lessons learnt Annexes

Ethical Considerations

All evaluation activities will adhere to ethical standards, including participant confidentiality and data protection in compliance with applied laws in the targeted countries.

Reporting and dissemination

The final evaluation report and summary presentation will be shared with stakeholders across Kosovo, Serbia, and Bosnia and Herzegovina. The findings will inform future project planning and policy discussions.

Necessary Skills and Capacities of the Evaluator

For Individuals:

• At least 5 years overall professional experience as an evaluator.
• Proven experience in conducting evaluations across multiple countries.
• Proven experience with logical framework approaches and other strategic planning approaches, M&E methods and approaches, information analysis and report writing.
• Track record in conducting project evaluations in education, civic participation and youth.
• Expertise in youth engagement, civic participation, and education.
• Strong background in participative data collection techniques.
• University degree in education, social sciences, international development, or a related field.
• Knowledge of the regional context in Kosovo, Serbia, and Bosnia and Herzegovina.
• Fluency in English, with knowledge of local languages being an advantage.
• Strong analytical, reporting and writing abilities.
• Openness to change and ability to receive and integrate feedback.
• Excellent communication, flexibility, and organizational skills.

For Companies:

• Demonstrated track record in conducting international evaluations, particularly in youth engagement and civic participation.
• Team members with strong expertise in participative data collection and project impact assessment.
• Company portfolio including successful evaluations in the education and social development sectors.
• Regional experience in Kosovo, Serbia, and Bosnia and Herzegovina, with a deep understanding of the local context.
• Availability of multilingual staff proficient in English and relevant local languages.
• Strong project management and organizational capacity to handle complex evaluations across multiple countries.

Timeframe / Schedule

The assignment is expected to commence in mid-September 2024 and conclude by the first weeks of October 2024. The final report must be submitted no later than 18 October 2024.

Award Criteria

The selection will be based on:

• Quality of Offer (50%): Understanding of challenges related to youth participation, civic engagement, and regional context.
• Proposed Methodology (30%): Clear and effective approach to conducting the evaluation.
• Financial Offer (20%): Competitiveness of the financial proposal, including daily fees and additional costs.

Application Process

Interested applicants should submit:

• CVs or a copy of business registration certificate and fiscal number.
• A cover letter detailing their suitability for the assignment.
• Proposed work plan and summary of the methodology (up to 3 pages).
• A detailed financial offer.
• At least two references for similar work conducted in the past.

Applications must be submitted in PDF format via email to [email protected] by 17:00 hrs on 10 September 2024. Late submissions or those not following instructions will not be considered.