TERMS OF REFERENCE (TOR)

INDEPENDENT MID-TERM EVALUATION OF THE PROJECT:

Together Advancing Kosovo’s Training and Employment within the Education Sector (TAKE)

BACKGROUND INFORMATION

Caritas Switzerland (CACH) is an independent Swiss NGO, engaged in Switzerland and more than 20 countries worldwide, including Kosovo. Our mission is to prevent, combat and eradicate poverty in a sustainable manner. To achieve this goal, CACH employs a policy of equity: we support inclusive communities with opportunities for everyone regardless of their ethnic background, group affiliation, or gender. CACH has been working in Kosovo since 1999, focusing on social inclusion, rural development and rehabilitation. Based on the Strategy 2021-2025, our central themes are, Income, Climate Change, and Migration.

Overall, the CACH Country Programme for Kosovo is based on CACH’s International Cooperation Strategy, and it defines country-specific outcomes. Within the theme of Income (Employment & Capabilities, and Market System Development), CACH in Kosovo is committed to Early Childhood Education, specifically on capacity building of in-service and pre-service educators, as well as single parents, leading to easy employment, promotion, and professional development.

CACH´s engagement in the early childhood system in Kosovo has shown that, for preschools to be of high quality, more should be done to support the professional development of educators through vocational education and training (VET). Therefore, one of the crucial aspects of ensuring quality service provision in early childhood is the development of professional skills for educational practitioners through VET and adult learning approaches. In-service educators need opportunities for continuous professional development, while pre-service educators require support through enhanced teaching and learning programs at university level. Beyond the direct effects that the professional development of education practitioners has on quality education, with the functionalization of the licensing system for teachers, the training of educators also contributes to higher individual earnings.

For more than fifteen years, CACH has been building on effective methodologies for providing adult learning for practitioners engaged in the field of early childhood education, following current international trends and adapting those to the local context in Kosovo.

The TAKE project goal is to “Ensure quality training of early childhood professionals, strengthening employment in education and capacitating caregivers to enter the workforce”. The project envisages maintaining and reinforcing the collaboration with the Ministry of Education and municipalities, aiming to ensure quality and child-appropriate education. Through this approach, the previous investments and successes are rendered more sustainable and, in close cooperation with government actors, the services are extended to more marginalised groups such as the hospitalised children.

The project will make use of the Network of Educators (NoE), an initiative currently being supported by CACH. The NoE promotes the engagement of educators in continuous professional development through workshops, webinars, self-directed courses, and non-formal exchanges. The NoE also creates possibilities for the development of resources and campaigns to better support parents regarding their roles and responsibilities towards quality education and services in early childhood. Finally, the project will scrutinize opportunities for providing professional VET training to parents (mostly mothers whose children have enrolled in preschools), and so paving their way into entering the workforce. By conducting needs analysis studies and providing tailored VET, good practices for supporting parents to enter the labor market will be structured for future scaleup.

TAKE project is in line with the UN Sustainable Development Goals (SDGs), specifically SDG 08 (Decent Work and Economic Growth) which promotes sustained, inclusive, and sustainable economic growth, full and productive employment and decent work for all. As well as SDG 4 (Quality Education) which ensures inclusive and equitable quality education and promotes lifelong learning opportunities for all.

1. PURPOSE OF THE EVALUATION

The mid-term evaluation should explore TAKE’s work and achievements and support to assess progress so far and provide guidance on how to maximize the potential for achieving the intended results and improve learning in its remaining timeframe (2022-2025). Through the assessment of the performance, achievements and lessons learned to date, the evaluation will contribute to both – learning and accountability. This evaluation will adhere to the OECD/DAC[1] and the SEVAL[2] standards for evaluation. For all practical purposes, this TOR defines the overall scope of this evaluation.

The main purpose of the mid-term independent evaluation is to provide an independent assessment of the progress to date, through an analysis of the relevance, effectiveness, efficiency, and sustainability of the project. The evaluation should give furthermore a brief outlook on the potential future impact of the project.

The specific objectives of the mid-term evaluation are the following:

a) Assess the relevance of the project’s design, regarding country needs and how the project is perceived and valued by the target groups. Evaluate the relevance of the TAKE project to address the needs for quality training of early childhood professionals, strengthening employment in education and capacitating caregivers to enter the workforce within the 12 municipalities. It will also assess the relevance of the stakeholders targeted by the intervention and the methodologies and approaches to do so.

b) Assess the effectiveness of the TAKE project at achieving its objectives and provide clear insights about what has and has not worked so far and why, including unexpected results and factors affecting project implementation (positively and negatively).

C) Assess the efficiency in terms of value for money of the delivery of the TAKE outputs.

d) Briefly assess and give a general outlook on the potential impact and sustainability of the project.

e) Identify lessons and provide a set of actionable recommendations on how the project and the project coordination/management could be adjusted for further improvement and to strengthen the sustainability of intervention and the delivery of results.

f) Provide strategic recommendations for the different key stakeholders to improve the implementation of the project activities and attainment of project objectives.

Recommendations emerging from the evaluation should be strongly linked to the findings of the evaluation and should provide clear guidance to stakeholders on how they can address them. It is expected that the findings and recommendations of this mid-term review will help to identify any needed course corrections in the project’s approach and activities and bring valuable external reflections to help strengthen the project and complement the MEAL system of the project through an adaptive management modality.

2. BRIEF DESCRIPTION OF THE PROJECT INTERVENTION

The project is maintaining and reinforcing the collaboration with the Ministry of Education, Science, Technology, and Innovation and municipalities aiming to ensure quality and child-appropriate education. Through this approach, the previous investments and successes are rendered more sustainable and in close cooperation with government actors, the services are extended to more marginalized groups such as the hospitalized children. However, this project goes a step further and widens its scope by focusing more on the labour market and income in the long-run (quality education leads to higher income opportunities) and in the short-run aiming at keeping in-service educators in the labour market, while also enabling young pre-service educators to enter the labour market. While the focus is currently on the education labour market the project is expanding and enabling parents to enter the workforce through VET training, as their children are now taken care of.

To understand limitations, challenges and opportunities, studies are being conducted (problem analysis, labour market analysis etc) and at the same time, a pilot with 30 parents will be started. The aim is to gradually increase the focus on the integration of parents (depending of course on lessons learned and results of the pilot project), i.e., especially single mothers, into the labour market. As a result, the parents not only are aware of the added value of ECE and their role as a parent but are also enabled to work (time and skills), the educators provide quality education and can remain in their jobs/enter and the children receive a quality education, setting the basis at an early age first for the formal school period and later in becoming skilled workers and generating safe incomes (and the Kosovar economy also benefits).

The overall objective of the project is:

Ensuring quality training of early childhood professionals, strengthening employment in education and capacitating caregivers to enter the workforce.

This shall be accomplished by cooperating, supporting and involving all stakeholders parents/caregivers, educators, institutions/schools, local and central institutions, academia, and the Ministry of Education, Science, Technology, and Innovation as the responsible institution for policy making in ECE.

The three outcomes of the project are as follows:

• The offer and quality of training and collaboration in the early childhood education sector are improved.
• Mechanisms that stimulate the integration of in-service and pre-service educators in the

labour market is created and parents are enabled to enter the workforce.

• The early childhood infrastructure is expanded, including access for marginalized and

hospitalized children leading to opportunities for further employment.

First Outcome aims at capacitating the main actors involved in children’s upbringing and development through practice-oriented approaches and vocational education & training. Professionals working on this education level will be capacitated through digital courses in topics relevant to early childhood, alongside the development of digital competencies as a key required to be up to date. Moreover, the community will be strengthened through the Network of Educators and the ownership of the network will be entirely transferred to local partners. The intervention focuses on raising awareness and training parents to proactively contribute to children’s development. This will be organized through the Faculties of Education (FEs) and their professors. The cooperation with the FEs will be further strengthened to further enhance the quality of teaching and learning at the faculties’ level. The FEs as main actors in the early childhood system will be given more responsibility in driving and implementing change through enhanced capacities in research and development and strengthened exchange and cooperation with other layers of the early childhood system (i.e., in-service educators, children, MEDs). The creation of didactic centers within the faculties equipped with state-of-the-art didactic materials opens opportunities for higher engagement of professors and pre-service educators in partnering up and being closer to the needs at the ground level. Finally, the creation of a database on quality for preschool education contributes to the digitalization of processes for early childhood enabling data-driven decision making.

Second Outcome focuses on creating and strengthening mechanisms that stimulate the integration of in-service and pre-service educators in the labour market. At the same time, it creates mechanisms for the integration of parents (mostly single, whose children have been included in the preschool system) into the workforce through the provision of tailored VET. Furthermore, at the level of pre-service educators who will soon enter the labour market, this outcome will address the need at the faculties’ level to review and restructure the manuals and procedures on the pedagogical internships put in place around 20 years ago. By reviewing and restructuring the internship process, the roles of mentoring professors and educators will be strengthened ensuring a higher-quality internship for pre-service teachers (future educators). The staff of early childhood institutions is currently mostly comprised of nursery assistants, who lack professional education in early childhood education (ECE). By creating mechanisms for the requalification of teachers, the employed teachers will benefit from staying within the workforce, while nurses and assistants who lack proper qualifications can further advance their professional development. The parents (with a focus on single-headed households) of children newly integrated into the preschool will be supported with utilising good practices of vocational education training (VET) schemes to get employment thereafter. It has been shown by research that the integration of children in ECE institutions opens the way for parents’ integration into the labour market. Consequently, the intervention supports at the same time economically needier parents in VET training so that they can be easier integrated into the labour market.

Third Outcome focuses on supporting key actors including MESTI, 5 public FEs, MEDs and other key actors in creating adequate spaces for children to access and participate in preschool and pre-primary education. Through the interventions, the limited number of preschools will be tackled to ensure the inclusion of a higher number of children. Likewise, pre-primary classrooms need to be equipped with adequate furniture and toys that accommodate the needs and the work methodologies for young children. Access to learning and development should be granted to all children. An innovation of the proposed initiatives includes the work with the pediatrics hospital in Prishtina to create and test approaches for including the hospitalized children in learning and development activities. For the first time, the Ministry of Education has shown interest in providing institutional support to hospitalized children in terms of learning, development and growth. Consequently, the project works closely with the ministry and central hospital to provide adequate infrastructure and structured activities for the hospitalized children.

3. SCOPE & FOCUS OF EVALUATION / EVALUATIVE QUESTIONS

3.1 SCOPE OF THE EVALUATION

The mid-term evaluation will analyze all planned outputs and outcomes under the project within this period (please see Table 1), with particular attention to synergies between the components and contribution to ensuring quality training of early childhood professionals, strengthening employment in education and

capacitating caregivers to enter the workforce.

The evaluation will also discuss, throughout the evaluation questions, how the project is addressing its main issue (i.e., Income-quality education,) and the CACH cross-cutting themes including conflict sensitivity, gender, partner support and participation. The evaluation should help to understand if, how and why the project has obtained the specific results from output to potential impacts, how changes in the context have influenced progress, and what adjustments are needed.

Furthermore, the evaluation process will assess the first implementation phase (20 months) of the TAKE project activities conducted and implemented during the period 2022 – 2023 to provide improvements based on the findings from the field and support further project implementation recommendations/phase.

The evaluation experts/service provider is not allowed to share any information and/or evaluation results without prior notification to and approval through the CACH/TAKE project staff. The service provider will sign a copy of the Code of Conduct document and attach it as Annex to the service agreement.

Table 1: Scope & focus of the evaluation.

Regionally the following target municipalities are part of the TAKE project:

Table 2: Geographical coverage

3.2 EVALUATIVE QUESTIONS & FOCUS

Concerning key five DAC criteria for mid-term evaluations (Relevance, Effectiveness, Efficiency, Impact, Sustainability), the following questions are of special interest:

3.2.1 Relevance

• How appropriate and relevant is the TAKE project towards the alignment with the national policies on early childhood education, ages 3-6?
• Is the project relevant to the identified needs of beneficiaries (target group of the project)?
• What is CACH/TAKE project's position towards cooperation with relevant stakeholders/actors in early childhood education?
• How well is the project coordinated with other projects, like UNICEF and other INGOs?
• To what extent is the project relevant for the purpose to:
  a) Capacitate pre-service and in-service educators, professors, education officials and parents on contemporary early childhood education methodologies and digital competencies through practice-oriented self-learning courses, webinars, workshops, and Network of Educators events?
  b) Enhance the quality of early childhood education at the Faculties of Education through research & development and strengthened collaboration across pre-service and in-service educators, professors, and children.
  c) Advance key processes related to the employment of pre-service and in-service educators at the institutional level;
  d) Prepare pre-service educators and recent graduates for future employment;
  e) Increase access for children of all backgrounds to quality preschools and pre-primary education

through the equipment of newly created preschools by the Ministry of Education, Science, Technology, and Innovation and 50 existing pre-primary classrooms with inventory, playing materials and other didactic materials; f) Create and test innovative approaches to offer learning and development opportunities to hospitalized children.

• Has there been any change since the project was formulated that might have affected its relevance? If so, what are these changes and to what extent has the project managed to adapt to ensure it remains relevant?
  
  3.2.2 Effectiveness
• How modern/up-to-date is the pre-school classroom equipment provided by CACH? What can be improved there?
• To what extent has and will the TAKE project deliver on its outputs and outcomes at Kosovo regional level to date? Were there any positive unintended consequences or unexpected effects? In particular:
  q) Is the project on track to reach the intended outcomes and outputs?
  b) How effective has the TAKE Project been to improve the offer and quality of training and collaboration in the early childhood education sector?
  c)How effective has the TAKE project been in creating and activating mechanisms that stimulate the integration of in-service and pre-service educators in the labour market and enabling parents to enter the workforce? d)How effective has the TAKE project been on expanding early childhood infrastructure, including access for marginalized and hospitalized children?
  e) How effective has the TAKE Project been in engaging with key decision makers in a country to Early Childhood Education in policy and decision making?
  f) What are the early markers of changes among decision/policy makers that demonstrate that the TAKE Project is on its way to tackle existing barriers to make behavioral change constructively possible and capacity building of educators in the respective municipalities?
• For all the above questions, what are the factors influencing positively and negatively the effectiveness of the project?
  
  4. DELIVERABLES

5.1. An inception report (not more than 20 pages, excluding the annexes) – upon the review of available documents and an initial discussion with the project management, the inception report will be developed, including:

• Conceptual framework that will be used to undertake the evaluation.
• Own methodology proposed.
• Data required to answer the evaluation questions, data sources by specific evaluation questions (emphasizing triangulation as much as possible) data collection methods, and sampling.
• Selection criteria for locations to be visited.
• Selection criteria for individuals for interviews (as much as possible should include men and women).
• Detail the work plan for the evaluation, indicating the phases in the evaluation, their key deliverables and milestones.
• Set out the list of key stakeholders to be interviewed and the tools to be used for interviews and discussions.
• Set out the agenda for the stakeholder’s workshop.
• Set out an outline for the final evaluation report.
• Interview and focus group guides.
• Evaluation question matrix - attached as Annex III to this TOR.
  
  5.2 Preliminary Findings to be shared with the key stakeholders

The inception report should be approved by the Project Manager before proceeding with the fieldwork. More detailed instructions can be found in Annex II - Writing the inception report.

After completion of data collection or before sharing the draft report, the evaluator should present preliminary debriefing and findings to the key stakeholders. The evaluator will set the agenda for the half-day to one-day meeting. The presentation should provide a brief review of key results for each evaluation criterion. The workshop will be technically organized by the evaluation team.

5.3 First draft of Evaluation Report

The report should be no longer than 30 pages excluding annexes. The Project Manager and the CACH team will review the first draft of the Mid-Term Review (MTR) report, to ensure that it meets the required quality standards and covers all agreed components and contents of the MTR. Detailed comments and feedback on the draft report will be provided to evaluation consultant(s) and discussion may be held to provide clarifications as necessary. The draft review report will also be shared with all relevant stakeholders for additional feedback and input. The evaluator(s) should submit a comprehensive draft report consisting of major findings and recommendations for future courses of action.

5.4 The final version of the evaluation report

Will be produced by the consultant(s) based on feedback received on the draft report, incorporating comments received from CACH/TAKE, and other key stakeholders. The report should be no longer than 30 pages excluding annexes and an executive summary. The final draft report should be submitted within the given timeline with enough detail and quality.

The draft and final versions of the evaluation report must be in English (maximum 30 pages plus annexes) and will be developed under the following structure (Annex IV - Preparing the evaluation report).

Table 3: Structure of the evaluation report

4.1.1 Efficiency

• To what extent are the TAKE outputs in balance with the level of effort, time and resources spent?
• To what extent has spending aligned with the planned budget, and how does project delivery progress compare to the planned schedule?
• How much has the project management adapted to any changing conditions to improve project implementation efficiency?
• To what extent has the project built on existing agreements, initiatives, data sources, synergies and complementarities with other and previous projects, partnerships, etc. and avoided duplication of similar activities by other groups and initiatives?
• Which alternative approaches could have led to equivalent results at lower costs?
• How efficient were the training courses provided?
• How financially efficient is the school equipment purchased financially?
  

3.2.4 Impact

• To what extent does the project demonstrate the potential to make a serious contribution to achieving the overall goal by the end of the project's duration?
• What could be adjusted in order to steer better toward reaching the intended impact of the intervention?
• How consistent are the activities and outputs with the intended impact?
• What could be adjusted in order to steer better toward reaching the intended impact?

“Ensuring quality training of early childhood professionals, strengthening employment in education and capacitating caregivers to enter the workforce.”

3.2.4 Sustainability

• How far the project has supported sustainable existent structures or to support and build such structures?
• To what extent has the Network of Educators become a sustainable, long-term initiative for life-long learning for Teachers?
• How sustainable are the didactic materials and inventory used in pre-schools?
• How sustainable are the improvements and activities at the pediatric clinic?
• To what extent did the intervention impact the beneficiaries in regard to income and career development/promotion?

The evaluator is requested to give a general and forward-looking assessment of the Impact and sustainability question only.

5. EVALUATION METHODS AND PROCESS

The independent mid-term evaluation will comply with the OECD/DAC and the SEVAL standards for evaluation. The evaluation will apply a mixed methods approach, engaging with key stakeholders of the project at all levels during the design, fieldwork, validation and reporting stages.

The evaluator(s) is expected to develop an evaluation framework based on the suggested key evaluation questions above but may suggest additional questions or modifications. The inception report will be prepared as the first deliverable of the evaluation and will include an evaluation matrix[3] presenting how the key issues will be addressed, the data sources and the data collection methods that will be used for the evaluation and a set of criteria to rate the strength of the evidence collected.

All data collection tools are to be included as annexes to the final evaluation report. The link between evaluation questions, data collection, analysis, findings and conclusions must be made and set out transparently in the presentation of the evaluation findings. Conclusion and recommendations should be underpinned by a strong set of evidence.

The evaluation will seek the views of the range of stakeholders who have been engaged in the process to date[4] to conclude whether the project is on track and expected to realize its set objectives.

To collect the data for analysis, the evaluation will make use of the techniques listed below (but not limited to). The data from these sources will be triangulated[5] to increase the validity and rigour of the evaluation findings.

• Review of relevant documentation from the project (documents to be shared once the evaluator has been selected).
• Desk review: project design and strategy documents, activity documents, communications, research and publications.
• Key informant interviews and focus groups: with project staff, tripartite constituents, beneficiaries of the project and potentially other stakeholders and partners.
• Field in-depth interviews: the evaluation team is expected to meet project beneficiaries (men and women) to undertake more in-depth reviews of the project work and results.

The evaluator must indicate the selection criteria for individuals to interview.

The selection of the field visit locations should be based on criteria to be defined by the evaluator.

• Presentation of the preliminary findings to the key stakeholders in a workshop.

At the end of the fieldwork the evaluation team will present preliminary findings to the project's key stakeholders in a workshop to discuss and refine the findings and fill information gaps.

The methodology should include examining the interventions, specifically in the light of the project’s log frame, logical connection between levels of results, its coherence with external factors, their alignment with the CACH’s strategic objectives, and Agenda 2030 – SDG (08 – Decent work and economic growth and 4 - Education) and related targets.

The data and information should be collected, presented and analyzed with appropriate gender, ethnicity and where feasible in terms of disability disaggregation, even if the project design did not take those into account. A more detailed methodology for the assignment will be elaborated by the evaluator based on this TOR, in consultation with the CACH/TAKE team and key stakeholders in the inception report.

6. SCHEDULE

This assignment is anticipated to take place between 15.03.2024 - 30.5.2024. The total duration of the evaluation process is estimated to be 36 service days for national evaluator(s)/expert(s).

Table 4: Evaluation Timetable and Schedule

5: Proposed, indicative workdays for the evaluation team

7. EVALUATION TEAM / QUALIFICATIONS

The evaluation team should consist of at least two evaluation specialists/consultants, who will work jointly to achieve the expected results. Both specialists will work in close coordination with the TAKE Project Manager and the TAKE team. The project team will provide administrative and logistical support as needed.

Evaluation Lead: will be advised by the TAKE Project Manager, who is responsible for completing the following specific tasks:

• Briefing the evaluator on CACH evaluation policies, procedures and sharing of relevant documents for desk review.
• Initial coordination with the project team on the development of the field mission schedule and organizing the preliminary results workshop, as well as providing feedback to the inception report.
• Clarification of the Terms of Reference in the initial meeting.
• Circulating the first draft of the evaluation report for comments by key stakeholders and CACH, and incorporating feedback into the report draft.
• Ensuring the final version of the evaluation report addresses stakeholders’ comments (or providing an explanation if any have not been addressed) and meets CACH requirements, approvals and payment processes.

The Evaluator(s) should have the following qualifications:

• Education:
• Master’s degree in one of the following relevant fields: Social sciences, preferably in Education, Early childhood education, Development studies, Public Management, Management or other related to the field.
• Academic/professional background in M&E, programme management is an asset.
• Work Experience (for at least one of the team members):
• At least 5 years overall professional experience as an evaluator.
• At least 5 years of relevant professional work experience in education, early childhood education, teacher training, education, or performance/results-based budgeting.
• Track-record in conducting project evaluations.
• Experience working with international development partners, especially in Early Childhood Education, teacher training, education or public financial management.
• Experience in working with OECD/DAC or the SEVAL standards.
• Proven experience with logical framework approaches and other strategic planning approaches, M&E methods and approaches, information analysis and report writing.
• Extensive knowledge of, and experience in applying qualitative and quantitative research methodologies.
• Functional competencies:
• Strong interpersonal and communication skills.
• Strong analytical, reporting and writing abilities.
• Excellent speaking and presentation skills.
• Openness to change and ability to receive and integrate feedback.
• Language requirements:
• Excellent spoken and written Albanian and English language skills required. Other local languages, in particular Serbian, are considered a strong asset.
  8. BUDGET

This assignment foresees a total of 36 days, to cover the whole evaluation process.

The available budget foreseen for the evaluation process is EUR 11,000.00.

The following costs will be covered by evaluator (i.e., they need to be included in the offer, in addition to daily fees of each evaluator):

• Transportation cost in Kosovo.
• One day workshop and necessary translation/interpretation costs, including venue, refreshment and lunch)
• Translation costs

CACH/TAKE project will not pay any additional costs/expenses that may arise from the evaluation process except the financial amount presented and agreed on in the service agreement with the evaluator(s).

ANNEXES

• Annex I: Bidding Instructions.
• Annex II: Writing the inception report.
• Annex III: Evaluation question matrix (EQM).
• Annex IV: Preparing the evaluation report.
• Annex V: General conditions for procurement.
• Annex VI: Exclusion criteria.

[1] https://www.oecd.org/dac/evaluation/daccriteriaforevaluatingdevelopmentassistance.htm [2] https://www.seval.ch/en/standards-competences/standards/
[3] ANNEX III - Evaluation matrix for the inception report.
[4] Indicative list and contact of stakeholders who have been engaged in the process to date - to be shared once the evaluator has been selected.
[5] Triangulation - looking for at least three sources or applying at least three different methods. https://rise.articulate.com/share/oPu-uFZAE0-X9S6EnnyBJ_WrjrYiTCFh#/lessons/j3AG eK2ILpYzzRcvfbFwc6dew39ThVw