Mid-Term Evaluation Terms of Reference

Empowering Rural Economies in Agriculture (EREA) General information

Job title: International and local evaluation specialists for the Mid-Term Evaluation of EREA Programme

Programme title: Empowering Rural Economies in Agriculture, EREA Donor contract no.: 8191-00/2023

Donor: Austrian Development Cooperation (ADC)

Programme country: Republic of Kosovo

Remuneration type: Daily fee

Project Context and Background

The Empowering Rural Economies in Agriculture (EREA) Programme is being implemented in Kosovo, where agriculture plays a vital role in rural economic development. The sector contributes approximately 7% to the country’s GDP and employs a significant portion of the workforce, particularly in rural areas. However, Kosovo's agricultural sector faces persistent challenges that hinder its competitiveness and sustainability. Limited access to finance makes it difficult for farmers and agribusinesses to invest in modern production methods, while weak market linkages restrict sales opportunities. The adoption of climate-smart agricultural practices remains low, making the sector vulnerable to environmental changes. Women and non-majority communities also face barriers to economic participation, limiting their ability to benefit from agricultural growth.

The EREA Programme was launched on November 1, 2023, and will run until October 31, 2026. It is funded by the Austrian Development Agency (ADA) and implemented by Caritas Switzerland (CACH) in collaboration with local partners. The programme applies a Market Systems Development (MSD) approach, aiming to create systemic and sustainable improvements in Kosovo’s agricultural sector by strengthening market access, improving financial mechanisms, and promoting climate-smart practices.

EREA focuses on three key agricultural sub-sectors: berries, non-wood forest products, medicinal and aromatic plants (MAPs), and fruit and vegetable processing. The programme works closely with national associations, agribusinesses, financial institutions, and rural communities to address systemic constraints in these sectors. By facilitating contract farming agreements, improving access to innovative financing models such as invoice factoring, and piloting climate-smart technologies like fog irrigation and heat exchangers, the programme seeks to improve the resilience and competitiveness of Kosovo’s agriculture. A strong emphasis is placed on gender and social inclusion, ensuring that women and non-majority communities have greater economic opportunities in the sector. The programme also works to strengthen the role of national associations in advocacy, advisory services, and market facilitation. In its first year of implementation, EREA has made progress in developing sectoral studies, implementing pilot initiatives, and engaging key stakeholders. However, as the programme reaches its mid-point, an independent evaluation is needed to assess its effectiveness and relevance of the programme’s interventions, identify challenges, and provide strategic recommendations for the remaining implementation period.

The programme is implemented in partnership with the following national and international organizations:

• Caritas Switzerland (CACH) – Lead organization responsible for programme coordination and management.

• Initiative for Agricultural Development of Kosovo (IADK) – Providing technical expertise in agriculture and rural development.

• Research Institute of Organic Agriculture (FiBL) – Supporting organic certification and sustainable farming practices.

• National Berry Association “Mjedra e Kosovës” – Supporting berry producers with market access and sector-specific expertise.

• National Association of Fruits and Vegetables Producers “PePeKo” – Strengthening the fruit and vegetable processing sector.

• National Association for MAPs and NWFPs “Organika” – Facilitating market access for medicinal and aromatic plant producers.

• Women’s Civil Society Organization “EcoKos-Women” – Promoting women’s empowerment and inclusion in rural economic activities.

The mid-term evaluation will assess the extent to which the programme is achieving its intended objectives, focusing specifically on the relevance and effectiveness of its interventions. It will examine whether implemented activities align with the actual needs and contribute to income generation and employment opportunities for farmers, agribusinesses, and market actors, and whether these activities are being executed as planned. The evaluation findings will inform strategic decisions, guiding programme adjustments to enhance results and ensure maximum impact for the remainder of the implementation period.

Purpose and Objectives

The mid-term evaluation of the Empowering Rural Economies in Agriculture (EREA) Programme is conducted as part of the monitoring and evaluation framework agreed upon with the Austrian Development Agency (ADA). The primary purpose of this evaluation is to provide credible evidence and insights to inform decision-making and strategic steering for the remaining period of programme implementation (2025–2026) and inform the follow-up programme design.

The evaluation is being carried out at this point in time to determine if the programme interventions remain relevant, are being effectively implemented, and align with national agricultural policies and development strategies. It will inform Caritas Switzerland (CACH) and its implementing partners (IADK, FiBL, PePeKo, Organika, Mjedra e Kosovës, EcoKos-Women) as well as ADA, about the programme’s achievements and challenges. Findings will be used to adjust activities, optimize resource allocation, and enhance overall programme performance.

The specific objectives of the evaluation are to:

• Assess the relevance of programme interventions for farmers, agribusinesses, targeted market actors and communities, emphasizing gender equality and social inclusion considerations.

• Evaluate whether interventions are being implemented effectively according to plan.

• Identify how well the interventions align with Kosovo’s existing agricultural policies and complementary donor initiatives.

• Provide actionablerecommendations for programme adjustments to ensure continued impact and alignment with systemic change goals.

The evaluation will serve primarily a steering function, guiding strategic and operational decisions for the remaining implementation period (2025–2026).

Scope

The mid-term evaluation will cover all programme activities implemented from November 2023 up to the time of evaluation (mid-2025). Geographically, it will include the twelve municipalities targeted by the EREA Programme: Dragash/Dragaš, Graçanicë/Gračanica, Kamenicë/a, Novobërdë/Novo Brdo, Peja/Peč, Podujeva/o, Prishtinë/Priština, Prizren, Ranil(l)ug, Shtërpcë/Štrpce, Suharekë/Suva Reka, and Viti/Vitina. The evaluation will assess interventions related to growth, inclusion and climate adaptation and mitigation outcomes.

The evaluation will apply the OECD/DAC evaluation criteria of relevance and effectiveness and will specifically consider how programme interventions integrate gender equality, social inclusion, and environmental sustainability. The evaluation will be conducted between months 18 and 22 of programme implementation, ensuring that findings are timely, feasible within available resources, and able to meaningfully inform the programme’s remaining period (2025–2026).

Evaluation Questions

The two selected DAC criteria of relevance, and effectiveness serve as guiding principles for the evaluation. The evaluation team is asked to pay particular attention and to address the following questions explicitly and clearly, collecting evidence to substantiate findings:

Relevance

1. How relevant and appropriate is the EREA programme’s Theory of Change in addressing the underlying challenges and opportunities in the agricultural sector, including gender equality and social inclusion, and in guiding the programme’s strategic direction and interventions?

2. How well are the programme’s interventions aligned with Kosovo’s agricultural policies (e.g., Strategy for Agriculture and Rural Development 2022–2028), EU policies for agricultural development, ADA’s strategic priorities, and other development programmes to ensure complementarity and avoid duplication?

3. To what extent do programme beneficiaries (including smallholder farmers, agribusinesses, women, and non-majority communities) perceive the implemented activities as relevant in generating income and creation of employment opportunities?

4. To what extent do programme interventions align with global priorities1related to climate resilience, sustainable agricultural practices, and social inclusion?

Effectiveness

1. To what extent have EREA programme interventions been implemented as planned and achieved their intended short-term results for women, marginalized groups, and smallholders, considering the internal and external factors that have influenced their timeliness and effectiveness?

2. How effectively have the interventions strengthened market system linkages and contributed to advancing gender equality, social inclusion, and policy or regulatory changes at local and national levels?

Design and Approach

3. Considering the purpose, objectives, and scope of the mid-term evaluation, a non-experimental design is deemed most suitable. As no baseline or pre-intervention control group was established before the programme began, the evaluation will focus on identifying patterns of change that align with the programme’s logic model and theory of change. This approach will assess how the EREA interventions contribute to observed outcomes without a counterfactual comparison.

The evaluation will use a mixed-methods approach, integrating both qualitative and quantitative data collection and analysis methods. A mixed-methods approach ensures a comprehensive understanding of the programme’s relevance and effectiveness, allowing for robust analysis through triangulation of evidence.

• Quantitative methods will measure the reach and short-term results of interventions. These methods may include structured surveys targeting farmers, agribusiness representatives, women entrepreneurs, and non-majority communities, ensuring representation across different beneficiary groups.

• Qualitative methods will provide in-depth insights into stakeholder perceptions, contextual factors influencing implementation, and early indications of change. Methods may include semi structured key informant interviews, group interviews, and focus group discussions with programme staff, beneficiaries, implementing partners, local authorities, and relevant government institutions.

Data Collection Methods and Analysis

To ensure reliability, credibility, and triangulation of findings, the evaluation is anticipated to include the following data collection methods:

• Desk review of programme documents, progress reports, MRM System, initial gender, social inclusion, and climate change assessments, strategic documents, and relevant policy frameworks.

• Structured surveys to capture quantifiable data on beneficiaries' experiences, benefits received, and short-term outcomes of implemented interventions.

1This question refers to the alignment of EREA interventions with global priorities, including the SDGs, EU GAP III, the UN Sustainable Development Cooperation Framework, and the Green Agenda for the Western Balkans.

• Key informant interviews (KIIs) with implementing partners, local authorities, ADA representatives, and other relevant stakeholders to understand intervention relevance, design appropriateness, and effectiveness. Approximately 25–30 KIIs will be conducted.

• Focus group discussions (FGDs) involving beneficiary groups (including women and non-majority communities) to explore their perceptions and experiences regarding relevance and initial effectiveness of interventions. Approximately 6 - 8 FGDs will be carried out to ensure depth and quality of information.

The evaluator(s) will ensure careful preparation of interview guides, survey questionnaires, and discussion protocols, considering adequate sequencing, clarity, and alignment with evaluation questions. The evaluation will systematically integrate the Human Rights-Based Approach (HRBA) and address ADA’s cross-cutting themes of gender equality, social inclusion, and environmental sustainability. Special attention will be given to assessing how inclusively and equitably interventions have been designed and implemented, ensuring that programme benefits reach vulnerable and marginalized groups. Gender and socio-economic disaggregated data will be collected and analyzed to highlight differential effects on various beneficiary groups, informing recommendations for improved inclusivity and equity. The evaluation will be conducted in strict adherence to the ADC Guidelines for Programme and Project Evaluations, the OECD/DAC Evaluation Norms and Standards, and ADA’s ethical guidelines. Confidentiality, informed consent, impartiality, and sensitivity to all stakeholders will be ensured throughout the process. The approach will be participatory, transparent, and respectful of diverse perspectives, ensuring balanced, credible, and unbiased findings.

This methodological approach is designed to be practical, rigorous, and feasible given the available timeframe, resources, and programme context. Further methodological details and refinements will be developed by the external evaluator(s) during the inception phase, ensuring alignment ADA’s Guidelines2 for Project and Programme Evaluations.

Workplan:

The evaluation is scheduled to begin in May 2025 and will be completed by July 15, 2025. Below is the detailed work plan and timeline for the assignment:

2 Guidelines_for_Programme_and_Project_Evaluations_ADA_2020.pdf (entwicklung.at): https://www.entwicklung.at/fileadmin/user_upload/Dokumente/Evaluierung/Evaluierungs_Leitfaeden/Guid elines_for_Programme_and_Project_Evaluations_ADA_2020.pdf

The timeline ensures sufficient time for data collection, analysis, and validation while maintaining efficiency within the allocated working days. The evaluation team will work closely with stakeholders to ensure the findings are timely, relevant, and actionable.

Evaluation Management Arrangements

The mid-term evaluation will be managed by Caritas Switzerland (CACH) and approved by the Austrian Development Agency (ADA). The EREA Programme M&E Manager will act as the main focal point, facilitating coordination between the evaluation team and all relevant stakeholders throughout the process.

A Reference Group (approx. 9 members), including representatives from ADA, CACH and implementing partners (IADK, FiBL, PePeKo, Organika, Mjedra e Kosovës, EcoKos-Women) will provide strategic guidance, review key deliverables, and ensure findings are relevant for programme adjustments. The evaluation will adhere to OECD/DAC evaluation principles, ensuring independence, impartiality, and ethical standards. Evaluators will work objectively, maintaining confidentiality and following ADA’s Guidelines for Programme and Project Evaluations in line with international best practices. Requirements for the Evaluators

The evaluation team (an international evaluation specialist and local evaluation specialists are mandatory) must be independent and must not have been involved in the design or implementation of the EREA Programme to ensure impartiality and objectivity in the evaluation process.

The evaluation will be conducted by a team of consultants who will work collaboratively to ensure a comprehensive and contextually grounded assessment. The international evaluation specialist will act as the team leader and lead author of the mid-term evaluation report. The team is expected to be gender balanced and possess a diverse set of complementary skills and expertise relevant to the scope of the evaluation.

The team should combine the following qualifications:

Evaluation Expertise:

• Proven experience in conducting programme and project evaluations using OECD/DAC evaluation criteria.

• Strong expertise in mixed-methods evaluation approaches, including qualitative and quantitative data collection and analysis.

• At least 10 years of experience in international development, economic development, social and gender empowerment.

• The team leader should have at least 10 years of experience in project and programme evaluations.

• Significant knowledge of Market Systems Development (MSD) and market-oriented approaches.

• Extensive experience in qualitative data collection, including planning and conducting interviews and focus groups.

• Working experience in the Balkans is an asset.

Thematic and Geographical Expertise:

• Expertise in agriculture, rural development, value chain development, and climate-smart agricultural practices.

• Extensive experience with the Kosovo agricultural sector, including policies, market structures, and relevant actors.

• Understanding of regional and EU agricultural policies and standard.

Human Rights, Gender, and Social Inclusion Expertise:

• Experience in applying a Human Rights-Based Approach (HRBA) in evaluations. • Strong understanding of gender equality and empowerement, social inclusion, and environmental sustainability in agricultural and rural development programmes.

• Ability to integrate gender-responsive and transformative methodologies in evaluation design and analysis.

Team Composition and Language Requirements:

• At least one member of the evaluation team should hold a Master’s degree in agriculture or rural development.

• At least one evaluator must have strong knowledge of the Kosovo context and local language skills (Albanian and Serbian) to facilitate data collection and stakeholder engagement.

• The lead evaluator must have excellent written and spoken English, with strong skills in preparing high-quality reports and presenting findings to stakeholders.

These qualifications ensure that the evaluation team possesses the necessary expertise, methodological rigor, and contextual understanding to conduct a thorough and credible assessment of the EREA Programme.

Specifications for the Submission of Offers

Interested candidates or companies are invited to submit an electronic copy of their proposal by 19 May 2025, with the subject line “Mid-Term Evaluation of EREA Programme”, to [email protected] . Technical Offer

The technical proposal should provide a clear, structured approach to the evaluation and include the following elements:

• Understanding of the assignment, outlining the context, objectives, and key evaluation questions. • Proposed methodology, detailing data collection and analysis methods, and justification for their selection.

• Work plan and timeline, specifying key phases, deliverables, and estimated working days.

• Composition of the evaluation team, including roles and responsibilities of each team member.

• CVs of key experts, demonstrating relevant experience in above specified fields.

• Two examples of previous evaluation reports and references, showcasing expertise in conducting similar assignments.

Financial Offer

The financial proposal should be submitted as a separate document and include:

• Detailed budget breakdown, covering consultancy fees, travel costs, accommodation, and any other anticipated expenses.

• Estimated working days for each consultant, aligned with the proposed work plan.

• Total cost of the assignment, expressed in EUR, inclusive of all taxes.

Evaluation of Offers

Proposals will be assessed using a best value for money approach, based on the following criteria:

• Technical proposal (80%) – Quality of approach, methodology, expertise, and alignment with evaluation objectives.

• Financial proposal (20%) – Cost-effectiveness and budget feasibility.

Budget Range

The estimated budget range for this assignment is maximum 40`000 EUR.

Submission Details

All proposals must be submitted electronically to [email protected] no later than 19 May 2025, 23:59 CEST.

Any questions related to this call must be submitted to [email protected] no later than 09 May 2025. Responses to questions will be shared in a timely and transparent manner with all interested applicants.