Explanatory Memorandum to COM(2017)69 - Ex post evaluation report on the Fundamental Rights and Citizenship programme (2007-2013)

Please note

This page contains a limited version of this dossier in the EU Monitor.

dossier COM(2017)69 - Ex post evaluation report on the Fundamental Rights and Citizenship programme (2007-2013).
source COM(2017)69 EN
date 13-02-2017
EUROPEAN COMMISSION

Contents

1.

Brussels, 13.2.2017


COM(2017) 69 final

REPORT FROM THE COMMISSION TO THE COUNCIL AND THE EUROPEAN PARLIAMENT

Ex post evaluation report on the Fundamental Rights and Citizenship programme (2007-2013)


2.

TABLE OF CONTENTS


TABLE OF CONTENTS

3.

1 INTRODUCTION


4.

1.1 OBJECTIVE OF THE REPORT


5.

1.2 METHODOLOGY AND SOURCE OF INFORMATION


6.

1.3 OVERVIEW OF THE PROGRAMME


7.

2 RESULTS OF THE EVALUATION


8.

2.1 RELEVANCE OF THE PROGRAMME


9.

2.2 COHERENCE AND COMPLEMENTARITY


10.

2.3 EFFECTIVENESS


11.

2.4 SUSTAINABILITY AND TRANSFERABILITY


12.

2.5 EFFICIENCY AND SCOPE FOR SIMPLIFICATION


13.

2.6 EUROPEAN ADDED VALUE


14.

3 CONCLUSIONS



1INTRODUCTION

1.1OBJECTIVE OF THE REPORT

1 2 The Decision establishing the fundamental rights and citizenship programme (FRC) requires the Commission to submit an ex post evaluation for the period 2007 to 2013. This ex post evaluation was performed by an independent external evaluator assisted by Commission staff.


This report is based on that evaluation. The report is structured according to the main evaluation criteria and corresponding questions. These include relevance, coherence and complementarity, effectiveness, impact and sustainability, efficiency and scope for simplification, and European added value.

1.2METHODOLOGY AND SOURCE OF INFORMATION

This final evaluation is based on the following:

• an extensive review of the available documentation for 135 action grants and 34 operating grants funded by the FRC programme 2007-2013;

• a review of programme documentation, such as the founding decision, annual work programmes and calls for proposals for both grants and public procurement contracts;

• a review of other information available online — e.g. EU policy documents, websites, founding decisions on related EU programmes etc.;

• a quantitative analysis of the 169 projects (action and operating grants);

• an analysis of 71 responses to the online survey from FRC programme grant beneficiaries;

• the write-ups of 17 follow-up interviews with coordinators of projects/organisations receiving FRC grants 2007-2013;

• four scoping interviews with Commission officials.


1.3OVERVIEW OF THE PROGRAMME

The FRC programme was established by Council Decision 2007/252/JHA of 19 April 2007 (the ‘founding decision’) for the period 2007-2013 as part of the ‘Fundamental Rights and Justice’ programme.

Article 2 of the founding decision outlines four general objectives:

• to promote the development of a European society based on respect for fundamental rights as recognised in Article 6(2) of the Treaty on the European Union, including rights derived from citizenship of the Union;

• to strengthen civil society and to encourage an open, transparent and regular dialogue with it in respect of fundamental rights;

• to fight against racism, xenophobia and anti-Semitism and to promote a better interfaith and intercultural understanding and improved tolerance throughout the European Union;

• to improve the contacts, exchange of information and networking between legal, judicial and administrative authorities and the legal professions, including providing support for judicial training to ensure a better mutual understanding among such authorities and professionals.

The general programme objectives complement the objectives of the European Union Agency for Fundamental Rights established by Regulation (EC) No 168/2007, i.e. to contribute to the development and implementation of Community policies in full compliance with fundamental rights.

The FRC programme, as with other programmes forming part of the fundamental rights and justice programme, funds actions through the following mechanisms 3 :

• action grants in the form of co-funding (up to 80 % of the total costs) for specific transnational projects of Community interest presented by an authority or any other body of a Member State, an international or non-governmental organisation, which may run for a maximum of two years;

• operating grants providing financial support (up to 80 %) to the activities of non-governmental organisations or other entities pursuing an aim of general European interest;

• operating grants co-financing expenditure associated with the permanent work programme of specific organisations mentioned in the legal basis (the Conference of the European Constitutional Courts and the Association of the Councils of State and Supreme Administrative Jurisdictions of the European Union); and

• tendered contracts/public procurement for undertaking specific actions initiated by the Commission such as: studies and research; opinion polls and surveys; formulation of indicators and common methodologies; collection, development and dissemination of data and statistics; seminars, conferences and expert meetings; organisation of public campaigns and events; development and maintenance of websites; preparation and dissemination of information material; support for and management of national experts networks; and analytical, monitoring and evaluation activities, etc.

The total budget allocated for the FRC programme for its implementation period (January 2007 – December 2013) amounted to EUR 94.8 million (see Table 1 1 ).

15.

Table 11 Planned budgetary breakdown for the FRC programme (2007-2013)


YearAvailable Budget for Grants and Contracts
Projects (action grants)Operating grantsCommission initiativesTotal annual budget
Value (€)%Value (€)%Value (€)%Value (€ m)%
20075 505 00051.9 %2 100 00019.8 %2 995 00028.3 %10.60100 %
20085 800 00050.0 %2 100 00018.1 %3 700 00031.9 %11.60100 %
20099 200 00064.8 %1 300 0009.2 %3 700 00026.1 %14.20100 %
201010 195 00073.9 %1 400 00010.1 %2 205 00016.0 %13.80100 %
20119 290 00067.3 %1 000 0007.2 %3 510 00025.4 %13.80100 %
201212 235 00080.0 %1 000 0006.5 %2 065 00013.5 %15.30100 %
201310 900 00070.3 %1 000 0006.5 %3 600 00023.2 %15.50100 %
63 125 00066.6%9 900 00010.4 %21 775 00023 %94.80 m

FRC — Annual Work Programmes (2007-2013)


In the period 2007-2013, the Commission published:

• five calls for proposals for action grants: FRC/AG 2007, 2008, 2009-2010, 2011-2012, 2013;


• six calls for proposals for operating grants: FRC/OG 2007 financial year 2008, FRC/OG 2008 financial year 2009, FRC/OG 2009 financial year 2010, FRC/OG 2010 financial year 2011, FRC/OG 2012 financial year 2012, and FRC/OG 2013 financial year 2013.

The Commission also funded 111 Commission initiatives through public procurement procedures.

Table 1 2 summarises the number of different actions funded in each year of the programme implementation period.

16.

Table 12 Number of actions funded per year


2007200820092010201120122013TOTAL
Action grants182649 42N/S 4135
Operating grants3746010434
Public procurement 7191427131615111


Most of the grants (46 %) went to national NGOs, including NGO platforms and networks. The next largest number (19 %) of projects were led by a European network/platform/forum, followed by national authorities (e.g. ministries) (6 %), research institutes (6 %) and universities (6 %). Moreover, 5 % of projects were led by local authorities, 4 % led by other education/training institutes, 3 % by regional authorities, and 5 % by other types of organisations. Finally, 0.6 % were led by courts, public services and prosecution services.

For FRC action grants, awareness-raising, information and dissemination represent 24 % of the activities, followed by mutual learning, exchange of good practices, cooperation (22 %), analytical activities (20 %), support to key actors (15 %) and training activities (13 %).

For FRC operating grants, awareness-raising, information and dissemination are the main types of activity at 25 %, followed closely by support to key actors (24 %), mutual learning, exchange of good practices, cooperation (23 %), analytical activities (16 %) and training activities (10 %). 5

2RESULTS OF THE EVALUATION

2.1RELEVANCE OF THE PROGRAMME

The relevance of a programme is assessed in terms of the extent to which its actions logically address its objectives, the wider policy needs of the EU and the needs of the target audiences.

2.1.1The relevance of the programme’s actions to its objectives

To ensure that the programme achieved results consistent with the objectives, the Commission set priority areas of work in each annual work programme. The priority areas determined the focus of activities funded through the action grants, as applicants were requested to put forward activities falling within one or more of the priority areas. Each year, priorities could change according to the EU policy agenda and according to the practical needs in the different sectors.

The first annual work programme in 2007 introduced the priority focusing on the protection of the rights of the child following the adoption of the Commission communication Towards an EU strategy for the rights of the child. The aim was to better promote and respect children’s rights as they are listed in the United Nations Convention on the Rights of the Child, through activities such as awareness-raising campaigns and analysis of specific needs of children. This priority area fits within the broader objective of promoting fundamental rights. The consistent prioritisation of this area over the years is therefore relevant. 6

The 2007 annual work programme also made the fight against racism, xenophobia and anti-Semitism a priority. This is in line with the FRC programme’s general objective on this issue. The idea was to promote actions targeted at fighting stereotypes and racist attitudes, speech and violence through initiatives aiming at fostering mutual understanding and at developing cross-community approaches. In 2008 one more priority was added — the fight against homophobia — which aimed at better identifying and fighting homophobic attitudes and stereotypes. These two priorities were maintained until the end of the programme in 2013. 7

The founding decision does not mention the issue of data protection and privacy rights, yet the Commission already made this a priority area in the 2007 annual work programme. As with children’s rights, this priority area fits within the broader objective of promoting fundamental rights, and is therefore relevant to the objectives of the FRC programme. In 2011, an additional priority was set, to improve cooperation between data protection authorities. No further changes were made to the priorities in this area during the programme period. 8

The founding decision mentions, as a specific objective, ‘to inform all persons of their rights including those derived from citizenship of the Union, in order to encourage Union citizens to participate actively in the democratic life of the Union’. This is therefore a highly relevant area, and it was given priority throughout the programme period.

The 2008 annual work programme introduced a priority focusing on training on the EU Charter of Fundamental Rights, aimed at improving the knowledge and understanding of legal, judicial and administrative authorities and legal professionals on the principles enshrined in the Treaty on European Union and in the EU Charter of Fundamental Rights. This priority was maintained until the 2012 annual work programme, in which priority was also given to educating the general public, with a specific focus on projects aimed at directing individuals who believed their fundamental rights had been violated towards the appropriate authorities. 9


17.

2.1.2Priorities set in the calls for proposals and selected actions, and their relevance to the policy


The priorities established in the FRC programme and set out in the calls for proposals were, overall, consistent with EU policy developments in the course of the programme, and appear to have adequately addressed those policy developments. 10 The programme’s priorities were mainly selected by the policy unit and according to the policy agenda and practical needs identified in each sector. The entire hierarchical chain was involved and different DGs could be consulted. This process was put in place to make sure that the priorities were consistent with the EU policy development.

For example, from 2010 one focus of the EU policy agenda with regard to citizenship was the promotion of gender balance in participating in the European Parliament elections. The following year, this issue was reflected as one of the FRC annual work programme priorities. 11


18.

2.1.3Relevance of the programme to the needs of the target groups


The FRC was a popular programme with quite a high number of applications received relatively to the available funding. The total number of applications in fact varied from a minimum of 66 in 2007 to a maximum of 257 in 2011-2012 for action grants. For operating grants the lowest number of application was 11 in 2007 and the highest number of applications was 52 in 2010. 12

The majority of respondents to the online survey had received more than one grant under the FRC programme, and they considered the priorities identified by the Commission in the calls as relevant to both the specific needs of the target group (80 %) and to the needs of their home country (86 %). 13 Also, most of the people interviewed (10 out of 17 interviews) were satisfied with the priorities as set in the calls for proposals, whilst only two out of 17 interviewees were not satisfied with the calls. 14

However, whether the needs of the target group could be met by the project activities, outputs and results and were actually met, could not be verified. Needs assessments are not a mandatory requirement for the grant beneficiaries. However, if grant applicants had done so, this would have given a much clearer indication of the relevance of their project, enhanced the quality of their study and provided a baseline for assessing project success at a later date. Projects that are not based on needs assessment may still be relevant to the target groups’ needs, but not producing a needs assessment creates a risk that more relevant methods or means of supporting the target groups could have been developed.


2.2COHERENCE AND COMPLEMENTARITY

The FRC focus on the protection of fundamental rights in general includes protection of the rights of children. The Daphne programme targets children among the vulnerable groups it aims to protect against violence. The Prevention of and Fight against Crime (ISEC) programme also refers to the protection of children among its general objectives, and in particular to protecting children from becoming victims of crime. In their focus on refugees and displaced persons, the European Refugee Fund and the European Integration Fund also include the protection of children among their aims, albeit those children who are among their target groups.

The objectives of combating racism, xenophobia, anti-Semitism and homophobia, which are part of the FRC’s remit on the protection of fundamental rights in general, are shared by the PROGRESS programme and the Seventh Framework Programme, among others. PROGRESS, which aims at the protection of persons from discrimination, has a narrower focus in that it is focused on contributing to the economic and social objectives set out in the Europe 2020 strategy. Daphne’s focus on ensuring protection from violence against certain vulnerable groups also has complementarities with this area.

The most scope for complementarity and potential for overlap was identified for the objective of training and networking between legal professions and legal practitioners. The FRC programme’s objective in this area is very broad, i.e. to improve contacts, exchange of information and networking between legal, judicial and administrative authorities and the legal professions. This included support to judicial training, with the aim of better mutual understanding, though focused on training related to fundamental rights in general. The scope for complementarity and overlap is the widest with the Directorate-General Justice the Criminal Justice Programme (JPEN) and Civil Justice Programme (JCIV). The JPEN, JCIV and FRC programmes aim at improving training for the members of national judiciaries. JPEN funds, however, can be allocated only to activities in the area of criminal justice, while JCIV funds can be allocated only to activities in the area of civil justice. Therefore JPEN and JCIV are more focused than FRC.

In conclusion, the most scope for complementarity was identified for the JPEN and PROGRESS programmes. Potential complementarities were also identified with the Daphne and ISEC programmes. The analysis identified no overlap with CIPS (Terrorism & other Security-related Risks), the RF (European Return Fund) or the DPIP (Drug Prevention and Information Programme). 15

19.

2.2.1Nature of the projects


The FRC covered only projects that were transnational in nature; national activities were covered only in so far as they were part of a larger transnational action. The FRC was implemented through direct management, along with the other funding programmes in place to support the EU policies on justice, rights and equality for 2007-2013. Under direct management, all programming and operational work is carried out by the Commission, which retains full responsibility.

20.

2.2.2Actions funded


The process of developing annual priorities within the FRC annual work programme is one mechanism for ensuring coherence between the FRC and other EU programmes and policies. These programmes, and in particular the annual priorities, are developed with the support of various actors, within and beyond the Commission. In this respect the programme reflects a range of policy priorities in the development of annual priorities.


A review of the priorities, set for the key thematic areas, identified the following instances where priorities were set with a view to achieving coherence and complementarity and avoiding overlap: 16


• Protection of the rights of the child: the 2008 and 2009-2010 calls for action grant proposals mentioned support for projects on children who are victims of crimes, which is complementary to the Daphne programme.


• Combating racism, xenophobia, anti-Semitism, and homophobia: the 2008 and 2009-2010 calls for action grant proposals similarly mentioned support for actions fighting stereotypes whose persistence or diffusion are at the roots of violent incidents, which is partly complementary to the Daphne programme.


• Under the priority ‘Fight against Homophobia’, the 2008 call excluded actions that duplicate activities on non-discrimination in employment. The 2009-2010 call had a similar exclusion. These are both aimed at avoiding overlap with PROGRESS.


• Citizenship and active participation in the democratic life of the Union: the priorities set in the various calls for proposals range from awareness-raising on participation in EU elections by EU citizens resident in another Member State to exchanging best practice on acquisition and loss of EU citizenship. Across the various calls for proposals this priority area does not appear to afford many opportunities for complementarity.


• Training on the EU Charter and networking between legal professions and legal practitioners: the 2011-2012 call excluded projects aimed mainly at the creation of new websites, in order to avoid overlap with projects already being carried out, presumably through public procurement. The 2013 call specifically prioritises projects to encourage authorities handling complaints on fundamental rights to draw up admissibility check lists, and requires project proposals to state their complementarity and added value with work carried out by the FRA.


• Data protection and privacy rights: under this priority area, the 2011-2012 call refers to ‘reinforcing children’s privacy in the online environment’, which forms an internal complementarity with the priority area ‘protection of the rights of the child’.


21.

2.2.1Synergies created between projects


The online survey revealed that various synergies had been established with other projects funded by other EU and national programmes. 17

Out of 29 respondents, 14 mentioned that they had established synergies with other FRC projects. Moreover, 21 reported that they had established synergies with projects funded by other EU programmes and 23 reported that synergies had been created with other national/regional programmes with similar objectives. Seven survey respondents stated that synergies with projects funded by other programmes of international donors were created.

Examples of the types of synergies established provided by respondents to the online survey included the following:

• The project leader for ‘Assisting and reintegrating children victims of trafficking: promotion and evaluation of best practices in source and destination countries (ARECHIVIC)’ noted that the project was able to link closely with projects funded under other EU programmes and with projects financed by international donors (e.g. the Council of Europe’s GRETA project). The project also established synergies with projects in other Member States, in particular with activities of the Slovak national commission. This was considered beneficial in establishing additional mechanisms for cross-border cooperation in this area. The leader of the same project also referred to synergies with projects financed through the ISEC programme and through the Norway Fund.

• The project leader for the ‘ICUD (Internet: Creatively Unveiling Discrimination)’ noted that during the implementation phase of the project they closely cooperated with leaders of other projects with similar objectives financed by FRC and DAPHNE III. The project leaders exchanged experience and information on the outcome of the projects, which enabled the project teams to learn from each other and to become more effective in combating this form of discrimination. The leader noted that members of the management team presented the project and its deliverables at conferences/seminars/meetings organised under other projects, and were able to disseminate their methodology and findings to a broader audience.

2.3EFFECTIVENESS

The effectiveness of a programme refers to the extent to which the programme has been successful in achieving its objectives, and the extent to which FRC projects were successful in achieving their own project objectives.

22.

2.3.1Programme and project achievements


Most the outputs produced under Children’s Rights (102 out of 143) related to awareness raising and analytical activities and were consistent with the aim of the priority (e.g. events, books or other published material and studies or guidelines on the promotion of children’s rights). 18 For example, one project entitled ‘Children’s voices: exploring interethnic violence and children’s rights in the school environment’ produced state of the art reports which gave an insight in a general context on ethnicity issues and interethnic violence in five Member States. Also the quantitative research surveys and the qualitative research field surveys provided information and an insight into the topic of interethnic violence in the school environment, forms of violence and the extent of the phenomenon.


The project ‘Right to Justice: Quality Legal Assistance for Unaccompanied Children’ produced an in-depth report on this subject, which had not been researched before. A ‘guiding principles’ tool was also developed. 19


The priorities focusing on Racism, Homophobia and Anti-Semitism were set to promote projects fighting stereotypes, discrimination and violence as well as initiatives fostering mutual understanding and developing cross-community approaches.


A number of action grants developed reports, training, and educational materials related to the fight against racism, homophobia and anti-Semitism. 20 An example of this type of project was the 2009-2010 action grant project by the German Autonomia Foundation entitled ‘One Europe! Transnational Network of Citizens Actions promoting Intercultural Understanding and Mutual Respect’. This project involved activities in over 20 localities in the participating countries (Denmark, Bulgaria, Hungary, and Germany) and aimed at bringing a message of tolerance to local youth and their sports teams. Also, the projects entitled ‘Football for Equality — Challenging racist and homophobic stereotypes in and through football’ and ‘Football for Equality — Tackling Homophobia and Racism with a Focus on Central and Eastern Europe’ produced outputs raising awareness on the issue of homophobia in football. These included ‘love football — hate sexism’ stickers; jerseys with the logo - ‘football for equality’; an anti-racism event in Prague; three information events in Hamburg; a comedy by the No Mantinels group dealing with homophobia in football; an annual tournament against racism and intolerance; an anti-homophobia conference at Eurogames, Budapest; and awareness-raising action days at Fans’ Embassies at EURO.


The initial aim of the Commission in setting a Data Protection priority was to promote activities informing the public about data protection issues. Over the years this priority area focused on reinforcing children’s privacy and on the risks to privacy posed by electronic means as well as on improving the cooperation between data protection authorities. According to the quantitative analysis of the 169 actions carried out under the FRC programme, the activities implemented by the projects under this area were mainly awareness-raising activities (e.g. published material, promotional material and events) and research activities such as data collection and surveys. Other outputs related to mutual learning activities, such as study visits and workshops. The project entitled ‘Données personelles des droits? Sensibiliser et informer les jeunes citoyens européens’ produced a good quality output — a comic in four languages — in order to disseminate information to young people about their rights to privacy and data protection online. The outcomes of the project are also considered valuable. 21


Also, the project entitled ‘Children Protecting Personal Data and Privacy’ created ‘Sheeplive’ cartoons, an online pedagogical tool for primary school teachers to educate children on their data protection and privacy rights. These are nationally recognised in Slovakia as the best known cartoons, have been translated into all EU languages and have been disseminated to Russia and China. 22


The Citizenship priority area includes a set of priorities focusing on participation in the democratic life of the Union by all citizens, including EU citizens who are not nationals of the Member States in which they live. Over the years more specific priorities were added, e.g. one addressing gender imbalance in European Parliament elections. It is a priority area characterised by a focus on very specific and sometimes technical issues (e.g. awareness of the rules in Directive 2004/38/EC on free movement of citizens) that might require specific and rather technical knowledge. This factor might explain the rather low number of projects carried out (21) and the low evidence of results. 23


The activities carried out in this priority area were mostly based on awareness raising, information and dissemination. For example, the project entitled ‘Citizens IT Consular Assistance Regulation in Europe’ produced good quality outputs including a comparative study on the consular protection laws and practices in the Member States and a database containing relevant national, European and international legal material. 24


Over the years, the aim of the Fundamental Rights priority was to improve the knowledge and understanding of legal, judicial and administrative authorities and legal professionals and of the general public on the principles enshrined in the Treaty on European Union and in the EU Charter of Fundamental Rights. Project activities in this area were primarily based on training and on mutual learning, exchanges of good practice and cooperation, aimed at achieving better mutual understanding among authorities and professionals. 25


2.3.2The projects’ contributions to the implementation and development of EU and national policies and legislation

A number of actions funded under the FRC programme contributed to the development of policy and legislation. 26 Some projects were specifically aimed at reaching policy makers at national and EU level, by different methods. For example, by organising meetings to involve policy makers, by inviting them to workshops and/or other dissemination events or inviting them to briefings/conferences, and — according to project partners — policymakers were reached. Indeed, a large majority of survey respondents (60 out of 65) reported that policy makers responded to the information provided by the project/activities. Out of these, 60 % reported that policy makers had shown some interest in the project while in 32 % of the cases they had shown a lot of interest.

Positive responses by policy makers took a range of forms. Survey respondents reported unanimously that policy makers had participated in project-related events. Other involvement of policy makers included distribution of communication materials (reported by 59 % of respondents) and providing additional funding to support continuation of (some) of the projects’ activities (reported by 30 % of respondents). A smaller but nonetheless significant proportion of respondents (18 %) noted that their projects’ results had led to policy makers shaping new policy developments or making adjustments to existing policies. In these cases the projects could certainly be considered as having made an effective contribution to the development of policy and legislation. 27

It is also noteworthy that 51 % (out of 70) of respondents reported that they had received positive attention from policy makers which had been additional to their expectations. This additional attention indicates that certain projects had an impact greater than originally foreseen, such that they could be considered effective. 28

23.

2.3.1Effectiveness of the projects in achieving their own objectives


According to the quantitative evidence gathered during the mapping of the final reports, over half of all finalised action grants and operating grants (70 out of the 115 grants for which final reports were available, since some projects awarded in 2012 and 2013 did not produce yet a final report by the time of this evaluation) implemented all their activities as planned. In some cases, achievement of this entailed changes to staff, or replacement of a member in the partnership or required searching for additional funding. Nonetheless, these fluctuations had a minor impact on the development of the funded actions (in fact, 63 % of the 71 respondents to the online survey indicated that these changes had no impact on the outcomes of their actions, while major impact was reported only in five cases). 29

For example, in the 2007 action grant project ‘ARIES — Against Racism in Europe through Sport’, although one of the original partners had to be replaced by another organisation and although changes were made to the project staff, all of the project’s objectives were achieved.

Another project result involved the training of law enforcement authorities. The project manager for the 2010 action grant project entitled ‘Fundamental Rights Education in Europe — F.R.E.E.’, in a follow-up interview, stated that her project had made a difference on human rights education of police officers, prison guards and judicial authorities. In her opinion the impact of such training was relevant especially on police officers in Greece, Italy and Romania, where — according to the jurisprudence of the European Court of Human Rights — cases of violation of human rights by law enforcement authorities are recurrent. Yet in these countries, police officers had never received training on such issues before. 30

2.4SUSTAINABILITY AND TRANSFERABILITY

In terms of sustainability, it was analysed whether the results, outcomes and impacts achieved by the projects were sustainable beyond the project funding period. Three levels of sustainability were identified in the evaluation:

1) short-term sustainability, achieved mainly through dissemination of projects’ results;

2) medium-term sustainability, implying continuation of project results and/or partnerships;

3) longer-term sustainability, achieved mainly by transferring projects’ results to other contexts, organisations and Member States with little or no additional funding. 31

24.

2.4.1Short-term sustainability: dissemination of project results


As noted earlier, the main types of activity covered by both the action grant and operating grant-supported projects are related to awareness raising, information and dissemination. Indeed, 346 of the total of 806 outputs tallied during the project mapping exercise — the largest share of project outputs — were linked to awareness raising and dissemination. Outputs included books, films and other published materials, as well as events such as conferences, seminars, and press conferences and support and advice services e.g. information and advice website and helplines. 32

Concerning the beneficiaries’ dissemination of outputs and results, the majority (89 %) of respondents to the online survey reported to have a clear plan for the dissemination of the outputs and results of their project/activities. The dissemination plans provided that the outputs and results would be disseminated in more than one language (93 % of respondents) and in more than one country (100 % of respondents). 33

Project partners reported having disseminated the outputs and results of the projects and increased their visibility by a range of means, including: seminars, conferences and other events, information leaflets, websites and electronic tools (e.g. videos and mobile applications), publication of reports and theme publications reflecting the conclusions reached in the activities and policy recommendations.

A review was carried out of selected applications to assess the quality of these dissemination strategies. These were then compared to what the grant beneficiaries said concerning dissemination of the project results in their final reports. The descriptions in the AG applications varied from a rather uninformative description of what the organisation had achieved in the past in terms of media coverage, to one which stressed that one of its work streams was devoted to advocacy and dissemination and described in detail how dissemination would be occurring throughout the duration of the project. None of the applications reviewed provided details concerning how they would adapt dissemination to the target group and most of the corresponding final reports were vague about whether the target group was reached.

In general, the Commission’s efforts to disseminate the results of projects have been limited. The lack of effective means for the Commission to share information concerning the FRC programme was already mentioned in the mid-term evaluation, and this evaluation did not find any additional communication resources and measures put in place. In particular, the FRC programme lacks a dedicated way to share results of one project with other beneficiaries, so as to spread best practices. 34 The Commission mostly relied on its beneficiaries to disseminate the outputs directly to their target groups. Also, the Commission's human resources were reserved mostly to the financial management of projects.

The lack of a proper mechanism/plan to communicate and disseminate the results of projects funded under the FRC programme was confirmed by the Commission officials. 35 They noted two different groups who could benefit from such a dissemination mechanism:

• Stakeholders involved in activities related to the FRC programme objectives, including funding applicants or beneficiaries of the programme; and

• EU policy makers and particularly Commission staff, including those managing the FRC programme.

Each of these groups has a different need:

• Stakeholders could benefit from having access to information about the activities and successes of other projects, so that they could consider how to build on the best practices of others. This might require a more structured dissemination of project results.

• EU policy makers, including Commission staff, might find the analytical results, mutual learning and other aspects from the projects helpful in the effort of designing better policies, including for implementation of the FRC programme. For them, it could be useful to get information highlighting the overall results of the programme, and the results of key projects.

25.

2.4.2Medium-term sustainability: continuation of results


The mapping of the projects showed that just under half (81 out of 169 grants) reported that they had developed sustainable outputs that were used after the programme funding had ended, and seven reports stated that the partnerships established outlived the project. A number of projects had also produced outputs that continue to be used for present and future activities. 36 For example, the project ‘Exchanging good citizen participation practices for the promotion of an active citizenship in the European Union (ESPACE)’ developed several outputs including awareness-raising materials and activities in four languages, mutual learning initiatives and national networks which continued to be used after project completion. Also, the project ‘When the innocent are punished — Children of imprisoned parents, a vulnerable group’ developed several sustainable outputs including the publication of a report and a summary entitled ‘Eurochips’ which was distributed in four languages to at least 10 Member States and which continues to be used to promote the rights of children with imprisoned parents.

The survey confirmed that sustainable results were achieved for a good portion of the projects. Out of 72 respondents, 44 (61 %) reported that their projects made a significant difference to the thematic area they are working in. They also reported a high response from policy makers to the funded projects. They noted a 100 % rate of participation by policy makers in project-related events, such as seminars, conferences, workshops etc. In 59 % (33) of the projects surveyed, policy makers had responded by distributing communication material (e.g. brochures, leaflets, flyers etc.); 30 % (17) of the policy makers involved had provided additional funding to support continuation of (some) of the projects’ activities; and 23 % (13) of policy makers involved had established/supported other projects that used elements of the project (approach/method, one or more activities etc.). Finally, 18 % (10) of policy makers shaped a new policy development/action plan/legislation or by adjusting existing ones using as a basis the project’s outputs and results. 37

2.4.3Continuation of partnerships after the projects’ completion

While the mapping found direct evidence of only 7 out of 168 projects continuing the partnerships established, survey results were much more positive. They suggested that many of the projects enabled the establishment of partnerships or networks which continued after the projects’ completion. Out of the 71 respondents to the survey, 48 (68 %) reported that their partnerships did or will continue after completion of the project. 38

Stakeholders interviewed reported that the cooperation built between partners through the project helped set up a network which is still working together on new projects. For example, the project ‘Children’s Voices: Exploring interethnic violence and children’s rights in the school environment’ helped to set up a new international network of researchers dealing with issues of ethnicity and interethnic violence. Its aim was to raise awareness among educators, children and the general public. The partners involved in the project ‘Données personnelles des droits? Sensibiliser et informer les jeunes citoyens européens’ also reported they planned to continue using the network established by the project. 39

26.

2.4.4Long-term sustainability: transferability of project results


To identify and evaluate whether the project outputs and results were transferrable and indeed transferred to another EU Member State this evaluation relied only on the data from the online survey and follow-up interviews. These data were self-reported and are not fully objective evidence. They show that a large number of outputs produced could be transferred to other EU Member States, particularly materials which, once translated, could be disseminated and used in other countries for other projects with similar objectives. Methods of approach may also be transferred to serve other target groups and beneficiaries, particularly any ‘best practice aspect’ of the output, which could be used in other contexts and projects. In a number of cases, minor conditions affecting the transferability of the projects’ outputs were reported. These mostly relate to the need to translate the outputs in other languages, or to adapt the project’s methods or outputs to the country-specific or local context by consulting with local stakeholders. 40

Most survey respondents confirmed the transferability of their projects’ outputs to other EU countries. For example, the training course prepared in the context of the project ‘Fundamental Rights Education in Europe — F.R.E.E.’ can be used in all Member States as they are available online and they are copyright-free. They are available in the language of the country participating in the transnational partnership established for the project. However, translation into other languages would be required to ensure the wider use of the courses. 41

Another project entitled ‘Football for Equality — Challenging racist and homophobic stereotypes in and through football’ organised exhibitions and workshops on the issue of discrimination in the football sector. 42 Project partners from different Member States coordinated with stakeholders within the national and local networks/football groups active in this sector to set up the events. In the follow-up interview, the project manager stated that the projects’ outputs could be transferred to other Member States easily by coordinating with national and local stakeholders to better tailor the events to the country-specific context.

2.5EFFICIENCY AND SCOPE FOR SIMPLIFICATION

27.

2.5.1Money spent in comparison to the impacts achieved


The FRC programme had an initial budget of EUR 94.8 million over the period 2007-2013, broken down into an annual provision of between EUR 10.6 million and EUR 15.50 million. A detailed breakdown for the initially planned budget per implementation year and funding tool for the period 2007-2013 is provided in Table 1 1 Planned budgetary breakdown for the FRC programme (2007-2013) ).


Over the course of the programme the total amounts of annual funding allocated gradually increased. This reflected the general practice of the Commission. The yearly allocation of funds is determined at the beginning of the seven year programme period, within the context of the EU budget. Small adjustments are then made annually, with allocations becoming higher towards the end of the programme period, once the programme is well underway.


The largest proportion of the budget was distributed via action grants with an average of 66.3 % of funds allocated in this way each year, corresponding to EUR 62.6 million. The remainder of the budget was allocated to operating grants (10 % or EUR 9.5 million) and to Commission initiatives (21 % or EUR 20 million). 43


The quantitative analysis of the 169 grant projects included over the implementation period (which does not include the projects awarded grants under the 2013 action grants) found that the total budget committed to these was EUR 46 460 022, or some 90 % of the total funding of EUR 51 675 000 that had been allocated to action grants for the period 2007 to 2012. 44


Whilst the amounts committed to action grants are close to the original allocations, the situation is different for operating grants, which received significantly less funding than initially envisaged. The budget actually committed to operating grants was EUR 3 459 900, or some 39 % of the total funding of EUR 8 900 000 that had been allocated for operating grants for the period 2007 to 2012. 45 The low funding levels of OGs raises questions about the efficiency of the OGs as a funding tool to achieve the objectives of the programme.


The budgets allocated, committed and paid to date are quite closely aligned for the action grants over the various calls for proposals during the programme period. The funds actually paid out, compared to those committed, under the 2007, 2008 and 2009-2010 calls for action grants show an absorption rate of some 87.5 %.


As already noted above, only 39 % of the funding allocated for operating grants during the programme period was actually committed. But a closer look at the commitments made during the 2007, 2008, 2009, 2010, 2012 and 2013 calls for operating grants, compared to the funds actually paid out, shows a similar absorption rate of 88 %. 46


The other funding tool available under the programme was public procurement. During the implementation period EUR 12 595 652 was actually committed, for a total of 111 public procurement contracts awarded during the programme period. This comes to 56.4 % of the total allocated budget of EUR 22 325 000. Over 10 % of this budget was committed to IT services and more specifically the development and maintenance of the consular protection website 47 and of the website on the rights of the child. 48


28.

2.5.2Extent to which the allocation of funds among different tools was efficient


Considering the above, action grant funding appears to have been allocated in an efficient manner, especially when taking into account the ‘newness’ of the programme, the outputs and results achieved and the findings of the input/output analysis above. Operating grants received considerably less than initially allocated (EUR 3.5 million instead of EUR 9.9 million), but budget absorption of operating grants was overall acceptable. As for action grants and operating grants funding allocations appear to have been efficient. 49


While each of the funding tools had a clear focus, procurement could, to some extent, overlap with the activities undertaken by these grants. During the implementation period, the Commission committed EUR 12.6 million on a total of 111 procurement contracts (or 56.4 % of the initially allocated budget of EUR 22.3 million). Some 55.6 % of expenditure on procurement was committed to studies; 34.2 % on events, and 10.2 % on IT-related contracts. Whilst the proportion of allocated funding actually committed is higher than that achieved for the operating grants funding tool, it still remains significantly under-used, which could have negatively affected the Commission’s interests in e.g. programme monitoring and dissemination of programme results at EU level. 50


Following stakeholder consultation, inefficiencies in relation to committing these funds mainly related to insufficient resources and capacity within the Commission to properly plan procurement, follow the implementation of projects and use their results. During this evaluation, no evidence of poor contract performance was reported. 51


29.

2.5.3Scope for simplification


Overall the management of the application and implementation phase of the projects by the Commission were considered positive. The introduction of kick off meetings, the involvement of the policy team in discussions on project outputs and the provision of improved guidance on managing grants were seen as positive developments. Nevertheless, 20% of respondents did not know or did not express their opinion. This could suggest that a number of applicants were not aware of the possibility to receive support from the Commission during the application procedure. Several interviewees highlighted areas that could be improved and simplified in the future. Chief among these areas was the perception that the financial requirements were overly complicated and needed to be simplified.


Linked to this was the concern expressed by a few respondents that the Commission itself did not always respect project timelines, which sometimes had knock-on effects on the project concerned, leading to difficulties in project implementation. It was seen as critical for the Commission to also stick to the timeline given for the project in order for everything to run efficiently. Linked to the previous suggestion, respondents also expressed frustration with the length of time between submitting a proposal in response to a call and receiving notification of a grant award. They asked for the Commission to create a more efficient and quicker way for people to contact them before the call is awarded, perhaps through improvements of the current IT system.


The vast majority of respondents to the on-line survey perceived the information provided to applicants during the call for proposals and application process as clear and easy to understand, and straightforward. Out of 71 respondents to this question, most grant recipients (50%) strongly agreed with the statement “the information in the calls for proposals was clear and easy to understand’’, while 44% partially agreed. In addition, detailed guidelines for FRC grant applicants were available on the DG Justice website.


Nevertheless, 18% of the 70 respondents to the online survey reported having to request help from persons with specific expertise and knowledge on the procedures in order to respond to the call, although 37% did not. A quarter of the respondents confirmed that they knew of organisations/projects/practitioners that did not respond to the call for proposals due to the complex/difficult requirements of the call.


In light of the fact that the EU is providing funds for these grants, grant beneficiaries considered it reasonable to provide information on the activities they had carried out with the funds provided. More than half of survey respondents considered reporting arrangements to be appropriate. In particular, 39% out of 71 respondents considered reporting arrangements concerning the progress and achievements of the project/activities to be appropriate, while 35% partially agreed, for a total of 74% positive responses. However, several complained about burdensome reporting requirements.

At the same time it was recognised that stricter documentation of budgetary changes would help with the financial reporting. It was also seen as important to have more involvement by partners in this area, e.g., by getting information to them early on concerning the financial details that would need to be reported.


As for the Commission’s monitoring arrangements, these were considered as partially good and helpful during the implementation of the project/activities by 39% of respondents out of 72 respondents. Nevertheless, 14% of respondents partially disagreed or even strongly disagreed with the statement ‘the Commission’s monitoring arrangements are good and have been useful’ and 5% strongly disagreed. Respondents considered it particularly important to get timely feedback from the Commission on projects’ progress/interim reports, so that corrections could be made if necessary.

2.6EUROPEAN ADDED VALUE

EU added value refers to the extent to which the EU nature of the programme brings value to its stakeholders and the extent to which the EU has a comparative advantage over national and international actors working in the area. First, the EU nature of the programme and its geographical coverage is discussed. This is a starting point for identification of a EU added value. The EU added value is then analysed in terms of outputs that brought value to the EU and to beneficiaries. However, it has not been possible to identify and measure EU added value already in terms of impacts.


In general, funding within the FRC programme is only available to activities with a clear European ‘added value’. The programme was created to promote the development of a European society based on respect for fundamental rights and rights derived from citizenship of the European Union. Such aims have implications that Member States cannot effectively address by themselves. The programme was therefore designed to require an international response comprising a coordinated and multidisciplinary approach, including the exchange of information at EU level and ensuring wide dissemination of good practices throughout the EU. Cooperation between different countries is therefore crucial for the implementation of the programme.


The EU nature of the programme is reflected in its legal basis outlining the objectives of the programme, as well as in the criteria to receive funding laid down in the calls for proposals. The founding decision states that one of the general objectives of the programme is to contribute to the development and implementation of ‘Community policies in full compliance with fundamental rights’. The EU dimension is further promoted via the criteria to receive funding and the theme of projects 52 , as laid down in the founding decision, the annual work programmes and the calls for proposals.


30.

2.6.1The geographical coverage of the projects funded



The number of lead organisations and thus the number of projects implemented differs significantly across Member States. For example Figure 2 1 shows that most FRC action grants projects were led by organisations established in Italy, followed by Belgium, France, Spain and the United Kingdom, whilst low numbers were found in e.g. Cyprus, Romania and Luxembourg. It follows that FRC projects have tended to be led by a select group of Member States, whilst other Member States were only involved to a limited extent.


Lead organisations were particularly clustered within two Member States: Italy and Belgium. In total 33 % (57) of all projects were led by these two Member States. Some Member States did not lead any FRC projects, such as Estonia. However, when looking at the partner organisations, the Member State participation is more evenly spread; out of all Member States, only Malta and Luxembourg participated with less than five partner organisations. 53

31.

Figure 21 Total number of organisations participating in FRC projects, including lead (left) and partner (right) organisations




Much of the FRC programme funding was committed to projects where an Italian organisation was a lead (25 %), followed by Belgium (13 %) and France (10 %). This funding distribution closely follows the distribution of the number of projects led by Member States. Distribution of funding by Member State of the lead organisation is presented in


Figure 2 2 (left). However, the figure does not show the spread of committed funding among project partners, but rather assumes that all of the committed funding was allocated to the country of the lead organisation. As this was not the case in reality (projects were transnational and project partners also received part of the funding) the figure should be interpreted with caution. 54


32.

The committed funding per Member State of lead organisation was further divided by population, to account for differences in Member State size (see



Figure 2 2 right). Assuming that the committed funding to lead organisations was not shared with partners outside the Member State of the lead organisation, then between 0.50 – 0.60 EUR per capita was committed in Belgium and Slovenia. Other Member States received less than 0.30 EUR per capita. 55


33.

Figure 22 Allocation of FRC committed funding by lead organisation (left) and by lead organisation per capita (right)





34.

2.6.2Structure of the partnerships



Figure 2 3 further highlights the partnership structure of the top three Member States with the highest number of lead organisations located in Italy, Belgium and France. The figures show that Italian lead organisations were more likely to partner with Italian partners (21 projects) followed by Spanish partners (14 projects) and Romanian partners (13 projects). This could reflect a preference for teaming up with partners in countries with similar languages. However, in total, Italy partnered with 22 different Member States.


Belgium lead organisations commonly partnered with organisations from the United Kingdom (eight projects), Spain (seven projects), Belgium (six projects), Greece (six projects) and Romania (five projects). In total, Belgium partnered with 24 different Member States.

France commonly partnered with Italy (four projects), Belgium (four projects) and France (four projects). In total France partnered with 14 different Member States.



35.

Figure 23 Partnership structure for the Top 3 Member States of lead organisations






36.

2.6.3Analysis of the geographical coverage of the programme


The FRC programme did not cover all Member States equally. The coverage of Member States largely reflects the number of applications received by different Member States. In total 804 applications were received for project funding through the FRC programme. The highest number of applications was submitted by Italian organisations (174) followed by organisations from Belgium (77) and France (69). This top three applicant Member States submitted 39 % of all applications for funding from the programme (see Figure 2 4 ). 56


37.

Figure 24 Total number of applications in FRC projects (right) and the success rate by MS (left)






The success rate of the submitted applications was the highest for Denmark and Ireland, followed by Finland, Austria, Belgium and Germany. Estonia and Malta applied but did not succeed in obtaining funding from the FRC programme. It should be noted again that these success rates were calculated based only on the applications for which Member States could be identified.


38.

2.6.4The added value for the EU and for grant beneficiaries


For grant beneficiaries responding to the online survey and follow-up interviews, the transnational partnerships brought specific benefits to their organisation, which no other funding source would have enabled on the same scale. Indeed, most survey respondents (88 %) stated that the project/activities would not have been implemented without EU funding. 57 Also, a significant part of activities developed under FRC would have not been developed had the programme not existed, since the programme itself had created a demand for new activities.


The partnership approach has resulted in an increased knowledge-base of participating organisations as evidenced by the survey results. Out of 59 respondents, the majority confirmed that the partnerships resulted in more knowledge/expertise in the topic area, enabled networking to consist of (more) international partners, and brought about more knowledge on policy and practice in other countries. The partnership also contributed, though to a lesser extent, to more knowledge on relevant EU policies and EU legislation. 58


3CONCLUSIONS

Relevance of the programme 59

• Overall, the priorities of the calls and the actions that were funded can be considered relevant to the programme objectives and to the development and implementation of EU policies and legislation.

• In general, the action grants and operating grants calls met the needs and interests of the main target group. However, some interviewees raised the need to increase the clarity of the calls.

• The priorities concerning the training of the general public or the judicial, legal and administrative authorities or legal professionals on the EU Charter of Fundamental Rights were not set consistently over the years.

• The robustness of the method used for the needs assessments carried out by the projects could not be assessed by this evaluation. Needs assessments are not a mandatory requirement for the grant beneficiaries. In the absence of needs assessments it is difficult to assess whether certain priorities should be dismissed or on what issues should be prioritised in the future.


39.

Coherence and complementarity


• The FRC programme is consistent with other European policy initiatives. Moreover, it is complementary with other EU funding programmes e.g. Daphne III, PROGRESS and JPEN, in terms of objectives and thematic areas and target groups. However, this complementarity also carries the risk of potential overlap with those programmes.

• Some projects funded under the FRC programme were complementary to the objectives and beneficiaries targeted by other EU funding programmes. Strength of the FRC programme is that it enables the financing of projects that may not fit into the narrower focus of other EU funding programmes.

• At project level, synergies were established with other programmes of international donors with similar objectives at the EU and national levels as well as with projects funded by the FRC programme itself.


40.

Effectiveness


• The actions funded under the FRC programme have made a notable contribution to the objectives it set out to achieve. In particular, the FRC programme has contributed to the implementation and development of EU policy and legislation.

• However, the data provided in the final reports and the interviews was not sufficient to enable an assessment of the specific level of effectiveness of these activities in achieving the programme’s objectives. For example, neither the final report template nor the guidelines providing instructions for completing the template gave a clear explanation of the difference between outputs, results and outcomes. Therefore, the final reports reviewed tended not to differentiate between the projects’ outputs, results and outcomes.

• The majority of the projects were able to achieve the planned results on time and to reach the expected target group. No obstacles seriously influencing the implementation of the project’s results or objectives were identified.

• The mechanisms in place for monitoring projects while they are under way do not ensure feedback is given to grant beneficiaries in all cases and or that lessons learned, good practices, success factors, etc. are identified.


41.

Sustainability



• The dissemination of project results by the grants beneficiaries was generally effective with some limitations. However, in the absence of continued monitoring by project partners once the project has ended, it is difficult to assess the long-term impact of their dissemination efforts. However, the Commission’s efforts to disseminate the results of projects have been limited. The lack of effective means for the Commission to share information concerning the FRC programme was already mentioned in the mid-term evaluation, and this evaluation did not find any additional communication resources and measures put in place.

• The FRC programme presents gaps in communicating and disseminating project outputs and results to EU policy makers and to other project managers that could benefit from those results. Sometimes projects’ material and tools required some minor changes, such as translation and adaptation of the project’s method to the local context, in order to be of use to other organisations addressing similar issues.

• Many of the projects’ outputs and results were considered transferable to other target groups or countries. In more than half of the projects the outputs have already been successfully implemented in another country.

• Overall, the FRC programme’s projects generated sustainable results in terms of continuation of activities and partnership building. Evidence that partnerships in a project continued after the project had ended was strong. However, whether project activities continued depended on the type of activity and whether continuation required further funding.


42.

Efficiency


• Overall, funding provided to action grants and operating grants appears to have been sufficient. Of the total funds allocated for action grants for the period 2007 to 2012, some 90 % was committed and of that, 88 % actually spent, indicating an acceptable absorption rate for these grants. For operating grants, only 39 % of the total funding allocated was actually committed, though of those amounts, the same percentage as above (88 %) was actually spent, again indicating an acceptable rate of absorption.

• The efficiency of the allocation of funds among the different funding tools was overall appropriate. The amounts available per project were regarded as sufficient for the implementation of the project’s activities and achieving the objectives.

• As regards scope for simplification, overall, the management of the FRC programme was efficient in terms of the requirements imposed on applicants and beneficiaries and the support received. However, some organisations encountered difficulties with the application and implementation process. Reporting requirements were sometimes considered as burdensome, particularly with regard to financial reporting. Overall, the Commission’s monitoring arrangements were considered useful; however some respondents experienced difficulties.


43.

EU added value


• The FRC programme has provided European added value. It has contributed to the development and strengthening of EU actions in the areas of freedom, security and justice and has particularly responded to the need to protect fundamental rights and promote EU citizenship. However, it has not been yet possible to identify and measure EU added value in terms of impacts.

• Projects covered a theme of relevance to the EU and were implemented by transnational partnerships between Member States. The transnational partnerships resulted in specific benefits, e.g. increased knowledge-base of participating organisations and a wider dissemination of good practices.

• The geographical coverage of projects and project partners receiving action grants and operating grants funding is uneven, with a few Member States overly represented and others quite under represented.


44.

Key recommendations


• Better define the priorities: the Commission should invest more time and human resources in the process of setting priorities in order to ensure that the priorities can be adequately achieved within an earmarked budget.

• Realistic assessments of project risks and better risk mitigation strategies: the Commission should better monitor risks throughout the project duration, for example by asking for brief progress reports that identify any potential risks as they arise during the implementation of the project.

• Increase focus on assessment of impacts at all levels and not merely on outputs, as regards monitoring and evaluation. This goes hand in hand with the need to collect, analyse and use objective and independent evidence in order to perform project and programme evaluations. Increase focus on needs assessment that each project aims to address.

• Explore ways of enhancing the uptake of project outputs, results and best practices by other organisations, including in other Member States, including more resources for translations, communication and dissemination.

• Sharpen the programme's intervention logic; further to the scope of the programme and its general and specific objectives and priorities, types of action and types of intervention and implementing measures, the Commission will seek to sharpen the intervention logic 60 , and make the relations between the rationale, objectives, inputs, outputs, beneficiaries, expected outcomes and impacts more articulate, precise and concrete in any future continuation of the programme.


(1)

Article 15(3)d, Council Decision 2007/252/JHA of 19 April 2007 establishing for the period 2007-2013 the specific programme ‘Fundamental rights and citizenship’ as part of the General programme ‘Fundamental Rights and Justice’.

(2)

The ex post evaluation report of the external evaluator is published here: ec.europa.eu/justice/grants1/files , Annexes: ec.europa.eu/justice/grants1/files,_2_and_3.pdf , ec.europa.eu/justice/grants1/files

The mid-term evaluation report of the Commission is published here: eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2011:0249:FIN:EN:PDF

(3)

Articles 4 and 8, Council Decision 2007/252/JHA of 19 April 2007 establishing for the period 2007-2013 the specific programme ‘Fundamental rights and citizenship’ as part of the General programme ‘Fundamental Rights and Justice’.

(4)

Not considered within the scope of this evaluation, because the final reports were not submitted during this evaluation process.

(5)

Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Fundamental Rights and Citizenship Programme (FRC), ICF, 28 July 2015, pp. 4-5 ec.europa.eu/justice/grants1/files

(6)

45.

Ibid., p. 7


(7)

Ibid.

(8)

Ibid.

(9)

46.

Ibid., p. 8


(10)

47.

Ibid., p. 11


(11)

Ibid.

(12)

48.

Ibid., p. 13


(13)

Ibid.

(14)

Ibid.

(15)

49.

Ibid., p. 15


(16)

50.

Ibid., pp. 16-17


(17)

51.

Ibid., p. 18


(18)

52.

Ibid., p. 19


(19)

Ibid.

(20)

53.

Ibid., p. 20


(21)

54.

Ibid., p. 21


(22)

Ibid.

(23)

Ibid.

(24)

Ibid.

(25)

55.

Ibid., p. 22


(26)

56.

Ibid., p. 23


(27)

Ibid.

(28)

Ibid.

(29)

57.

Ibid., p. 26


(30)

Ibid.

(31)

58.

Ibid., p. 28


(32)

59.

Ibid., p. 29


(33)

Ibid.

(34)

Ibid.

(35)

Ibid.

(36)

60.

Ibid., p. 30


(37)

Ibid.

(38)

61.

Ibid., p. 31


(39)

Ibid.

(40)

62.

Ibid., p. 32


(41)

Ibid.

(42)

Ibid.

(43)

63.

Ibid., p. 35


(44)

Data for 2013 were not available at the time of the evaluation. Ibid.

(45)

Ibid.

(46)

Ibid.

(47)

Available at ec.europa.eu/consularprotection/index.action , Ibid.

(48)

Available at ec.europa.eu/0-18/ , Ibid.

(49)

64.

Ibid., p. 40


(50)

Ibid.

(51)

Ibid.

(52)

Articles 4 and 9, Council Decision 2007/252/JHA of 19 April 2007 establishing for the period 2007-2013 the specific programme ‘Fundamental rights and citizenship’ as part of the General programme ‘Fundamental Rights and Justice’.

(53)

65.

Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Fundamental Rights and Citizenship Programme (FRC), ICF, 28 July 2015, p. 45


ec.europa.eu/justice/grants1/files

(54)

66.

Ibid., p. 46


(55)

Ibid.

(56)

67.

Ibid., pp.47-48


(57)

68.

Ibid., p. 48


(58)

Ibid.

(59)

69.

Ibid., pp. 50-51


(60)

See for instance Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Fundamental Rights and Citizenship Programme (FRC), ICF, 28 July 2015, pp. 1-2

ec.europa.eu/justice/grants1/files Error! Reference source not found.