Legal provisions of COM(2017)59 - Ex post evaluation report on the Civil Justice Programme (2007-2013) - Main contents

Please note

This page contains a limited version of this dossier in the EU Monitor.

dossier	COM(2017)59 - Ex post evaluation report on the Civil Justice Programme (2007-2013).
document	COM(2017)59
date	February 13, 2017

EUROPEAN COMMISSION

Brussels, 13.2.2017

COM(2017) 59 final

REPORT FROM THE COMMISSION TO THE COUNCIL AND THE EUROPEAN PARLIAMENT

Ex post evaluation report on the Civil Justice Programme (2007-2013)

CONTENTS

1    INTRODUCTION

1.1    OBJECTIVE OF THE REPORT

1.2    METHODOLOGY AND SOURCE OF INFORMATION

1.3    OVERVIEW OF THE PROGRAMME

2    RESULTS OF THE EVALUATION

2.1    RELEVANCE OF THE PROGRAMME

2.2    COHERENCE AND COMPLEMENTARITY

2.3    EFFECTIVENESS

2.4    SUSTAINABILITY, TRANSFERABILITY AND INNOVATION

2.5    EFFICIENCY AND SCOPE FOR SIMPLIFICATION

2.6    EUROPEAN ADDED VALUE

3    CONCLUSIONS

1INTRODUCTION

1.1OBJECTIVE OF THE REPORT

¹ ² The Decision establishing the Civil Justice programme (JCIV) requires the Commission to submit an ex post evaluation for the period 2007 to 2013. The ex post evaluation was performed by an independent external evaluator assisted by the relevant Commission staff.

This report is based on that evaluation. The report is divided into sections on the main evaluation criteria and corresponding questions. These include relevance; coherence and complementarity; effectiveness; impact and sustainability; efficiency and scope for simplification; and European added value.

1.2METHODOLOGY AND SOURCE OF INFORMATION

This final evaluation is based on the following:

• an extensive review of the available documentation of 130 action grants and 16 operating grants awarded under the 2007-2013 programme;

• a review of documentation such as the founding decision, annual work programmes and calls for proposals for both grants and public procurement contracts;

• a review of EU policy documents and the founding decisions of related EU programmes;

• a quantitative analysis of a sample of the 146 action grants and operating grants awarded;

• an analysis of 52 responses to the online survey from JCIV grant beneficiaries;

• 15 follow-up interviews with coordinators of projects/organisations receiving 2007-2013 JCIV grants ³ ;

• four scoping interviews with Commission officials.

1.3OVERVIEW OF THE PROGRAMME

The JCIV was established by Decision 1149/2007/EC of 25 September 2007 for 2007-2013, as part of the Programme Fundamental Rights and Justice.

Article 2 of the founding decision lists the general objectives. These are to:

• promote judicial cooperation with the aim of contributing to the creation of a genuine European area of justice in civil matters based on mutual recognition and mutual confidence;

• promote the elimination of obstacles to the good functioning of cross-border civil proceedings in the Member States;

• improve the daily life of individuals and businesses by enabling them to assert their rights throughout the European Union, notably by fostering access to justice;

• improve the contacts, exchange of information and networking between legal, judicial and administrative authorities and the legal professions, including by way of support for judicial training, with the aim of better mutual understanding among such authorities and professionals.

Article 3 lists the specific programme objectives. These are to foster judicial cooperation in civil matters aiming at:

• ensuring legal certainty and improving access to justice;

• promoting mutual recognition of decisions in civil and commercial cases;

• eliminating obstacles to cross-border litigation created by disparities in civil law and civil procedures and promoting the necessary compatibility of legislation for that purpose;

• guaranteeing a proper administration of justice by avoiding conflicts of jurisdiction;

• improve mutual knowledge of Member States’ legal and judicial systems in civil matters and promote and strengthen networking, mutual cooperation, exchange and dissemination of information, experience and best practices;

• ensure the sound implementation, the correct and concrete application and the evaluation of Community instruments in the area of judicial cooperation in civil and commercial matters;

• improve information on the legal systems in the Member States and access to justice;

• promote the training of legal practitioners in Union and Community law;

• evaluate the general conditions necessary to reinforce mutual confidence, while fully respecting the independence of the judiciary;

• facilitate the operation of the European Judicial Network in Civil and Commercial Matters established by Council Decision 2001/470/EC.

The JCIV’s specific objectives narrow down the general objectives to the expected outcomes of the programme. Like other programmes forming part of the Directorate-General for Justice’s General Programme on Fundamental Rights and Justice 2007-2013, the JCIV provides for various types of action, including:

Action grants for ⁴ :

• specific transnational projects of Community interest. These can be presented by an authority or any other body of a Member State, an international or non-governmental organisation, under the conditions set out in the annual work programme. These projects are co-financed up to 80% of the total costs and form the core of the programme.

• specific projects on the implementation of the European competition rules. These are projects aimed at promoting judicial cooperation between, and the training of, national judges who have to enforce European competition rules including Articles 101 and 102 of the Treaty on the Functioning of the European Union (TFEU) and the State aid rules, with the aim of contributing to developing and implementing European competition policy and Community cooperation measures on competition, to ensure the consistency of Community competition policy.

Operating grants intended ⁵ :

• to co-finance (up to 80%) the annual work programme of non-governmental organisations or other entities pursuing an aim of general European interest (calls for proposals);

• to co-finance expenditure associated with the permanent work programme of specific organisations mentioned in the legal base (the European Network of Councils for the Judiciary and the Network of the Presidents of the Supreme Judicial Courts of the European Union), provided that the grants are used to achieve an objective of European general interest;

• to fund the operation of the European Judicial Network in Civil and Commercial Matters established by Council Decision 2001/470/EC.

Commission initiated projects for conducting studies and research, opinion polls and surveys; devising indicators and common methodologies; collecting, developing and disseminating data and statistics; holding seminars, conferences and expert meetings; running public campaigns and events; setting up and maintaining websites; drafting and publishing information; supporting and running networks of national experts; and analysing, monitoring and evaluating the work done. The Commission uses procurement procedures ⁶ for these activities.

Table 1 1 shows the number of different actions funded in each year. Calls for proposals for action grants and operating grants were split across the years 2011 and 2012. The total planned budget for the JCIV for January 2007 to December 2013 was EUR 109.3 million ⁷ (see Table 1 2 ). Over the period, the Commission published 6 calls for proposals for action grants and 6 calls for proposals for operating grants, and 247 Commission initiatives were launched.

Table 11 Number of actions funded per year

	2007	2008	2009	2010	2011	2012	2013	TOTAL
Action grants	24	19	18	16	25	28	130
Operating grants	2	3	3	3	2	3	16
Public procurement	2	22	22	52	36	63	50	247

Table 1 2 gives a detailed breakdown of the budget for each implementation year and funding tool, based on annual work programmes. As the table shows, the programme includes specific projects on competition (EUR 800 000 a year, which is managed by the Directorate-General for Competition); the management of membership of the community in the Hague Conference on private international law; and operating expenditures of the European judicial network. The funding for the network covers about six general meetings per year to discuss problems in applying civil justice instruments, the publication of practical guides for practitioners and the public (including translation, printing and shipping costs) and updating the website (for which working groups are usually set up). All actions are decided the preceding year when the budget is being drawn up.

The budget allocated to action grants and operating grants between 2007 and 2013 was EUR 49.2 million, of which EUR 34.9 million had been committed and EUR 14.6 million spent as of January 2015. ⁸

Table 12 Planned budget breakdown for the JCIV (2007-2013)

Year		Available budget for grants and contracts
	GRANTS	PUBLIC PROCUREMENT
	AGs ⁹	AGs comp ¹⁰	OGs NGOs ¹¹	OGs other ¹²	HCCH ¹³	EJN ¹⁴	COM initiatives ¹⁵
2007	4 300 000	0	1 500 000	500 000	0	3 250 000	4 550 000
2008	4 500 000	0	1 500 000	500 000	0	3 150 000	4 650 000
2009	4 800 000	0	1 500 000	500 000	20 000	2 700 000	4 650 000
2010	5 050 000	0	1 500 000	500 000	20 000	3 050 000	5 380 000
2011	3 980 000	800 000	500 000	500 000	20 000	2 205 000	7 695 000
2012	4 350 000	800 000	500 000	500 000	34 000	2 700 000	7 216 000
2013	8 776 000	800 000	500 000	500 000	30 000	800 000	5 150 000
Total	35 756 000	2 400 000	7 500 000	3 500 000	124 000	15 155 000	39 291 000

Source: JCIV — Annual Work Programmes (2007-2013)

Most JCIV actions were led by a national authority (22% of all lead organisations), a university (21%), or a European network, platform or forum (20%). Compared to other programmes such as Daphne and Fundamental Rights and Citizenship Programme (FRC), national non-governmental organisations (NGOs) were in the minority, leading only 8% of JCIV actions. Universities (22%) and national authorities (20%) accounted for the bulk of partner organisations, followed by other education and training institutes (15%), national NGOs and platforms (13%) and legal professionals (10%). ¹⁶

Project awarded JCIV action grants mainly focused on analytical activities, training, and awareness-raising, information and dissemination (see Figure 1 1 , left).

Operating grants mainly went towards supporting key actors; mutual learning; exchange of good practices and cooperation; and awareness-raising, information and dissemination (see Figure 1 1 , right).

Public procurement contracts focused on studies, practical guides, conferences, workshops, maintenance of websites and other IT provision.

Figure 11 JCIV action grants projects by main activity (left) and JCIV operating grants by main activity (right)

Figure 1 2 provides an overview of the number of times a type of output was identified in the 146 JCIV projects mapped. ¹⁷ For example, the output ‘workshops and focus groups’, related to mutual learning and networking, which was identified 66 times and for which 59 indicators were found, had a total of 4 345 participants (the total of 33 indicators). The target groups were most often judges, practitioners and judicial staff.

The second most frequent output was training activities (42), as part of awareness-raising, information and dissemination. These accounted for 4 371 participants (based on 28 different indicators). The target groups were most often judges, mediators and practitioners.

The third most frequent output (41) was events, with a total of 3 646 participants (identified 23 times), and 8 000 recipients (the outputs of one indicator). The target groups were most often practitioners, the general public and judicial staff.

Figure 12 Count of JCIV outputs

2RESULTS OF THE EVALUATION

2.1RELEVANCE OF THE PROGRAMME

The relevance of a programme is assessed in terms of the extent to which its actions logically address its objectives, the wider EU policy needs and the needs of the target audiences.

2.1.1Relevance of the programme’s actions to its objectives

All general and specific objectives are geared to promoting judicial cooperation with the aim of contributing to the creation of a genuine European area of justice in civil and commercial matters. In assessing whether the objectives are SMART ¹⁸ , it can be concluded that they are specific (in that they target a specific area for improvement), attainable and realistic (as they can be achieved through the actions funded and within the programme resources). However, they are not measurable, as they do not quantify or at least suggest an indicator of progress. Nor are they time-bound, as the objectives do not have a time line, and it is not specified when the results should be achieved. ¹⁹

The Commission has a broad objective of fostering European judicial training, which prioritises training on specific subjects to meet policy needs and in line with the work of the European Judicial Training Network (EJTN). In 2011, the European Commission set a target of training half of all legal practitioners in the EU (around 700 000) in European law and in the law of a Member State by 2020 ²⁰ . Under the JCIV programme, European judicial training was mainly supported through action grants and a few specific actions, though operating grants were also awarded through other programmes ²¹ .

Given the Commission’s responsibility for running the secretariat of the European Judicial Network in Civil and Commercial Matters (EJN-Civil), funds were also allocated through procurement or administrative arrangements to support its operation. This pertains to a general objective, although it could be argued that it is a cross-cutting objective. The network generally meets to discuss problems in the application of the existing civil justice instruments, the publication of practical guides for practitioners or the public and updating the website. ²²

2.1.2Priorities set in the calls for proposals and selected actions and their relevance to policy

The JCIV programme has been closely linked to the main policy developments and legal instruments in the civil justice area from the start, and regular efforts were made to align the JCIV programme with new policy and legislative developments by setting priorities in the calls for proposals.

Overall the priorities and funded actions can be considered relevant to the objectives of the programme. However, the process put in place to select the programme’s priorities left little room for manoeuvre, once the priorities had been set (although in principle a project which did not reflect any priority could receive a grant under other award criteria). This meant that in some cases, if a priority changed during the implementation of a project, its results were less useful. Conversely, with regard to training, for example, needs could arise that did not fall under the set priorities. They could not then be funded under the programme because they were not taken into consideration in a specific call.

The programme was implemented:

• to contribute to policy and legislative developments, through research, analysis and other support activities, to provide the legislator with clear and detailed information on the problems and the situation on the ground, as well as through strengthening networks to assist with the preparation of future initiatives in this area. Among its most important achievements, the 2009 action grant project ‘Interconnecting European Registers of Wills’ aimed at developing a large network of interconnected registers of wills, in line with the policy development which finally led to the adoption of the Succession Regulation ²³ in 2012. Public procurement contracts were also largely used to that end, e.g. through the European Judicial Atlas in Civil Matters, the JURE case-law database, and various evaluation and implementation studies.

• to support the practical application and implementation of existing EU policy and legislation, through promoting networks, to help with the consistent implementation of EU instruments across Europe, through training and capacity building for legal professionals and other practitioners, to equip them with the tools to effectively put EU rights and policies into practice. Among its most important achievements, two 2009 action grant projects aimed at creating user-friendly tools —available in the European e-Justice Portal — to find civil law lawyers and notaries respectively across the EU. Another example is the ‘i-Support’ action grant, to explore ICT systems with a view to implementing the 2009 EU Maintenance Regulation and the 2007 Hague Convention on maintenance, accession to which was under discussion at the time the grant was awarded. Public procurement contracts played an important role here too, e.g. via stakeholder consultations, impact assessments and feasibility studies. ²⁴

2.1.3Relevance of the programme to the needs of the target groups

Overall, grant beneficiaries found that the calls and selected actions identified, and responded to, the needs of the target groups, although the extent to which they adequately identified these needs varied, with some undertaking elaborate needs assessments and others adopting a less rigid approach. Most grant beneficiaries consulted for this evaluation reported that they conducted needs assessments to support the design and development of their projects. However, analysis of the project documentation showed that a significant number of grant applicants either did not design their projects on the basis of a needs assessment or did not sufficiently document that assessment in the grant application form. While the Commission did not specifically require them to conduct and demonstrate needs assessment, these would have enabled grant applicants to make the relevance of their project much clearer, enhanced the quality of studies and provided a baseline upon which to assess project success later. Projects that are not based on needs assessment may still be relevant to target groups’ needs. But a needs assessment makes it more likely that applicants will pick the most relevant methods or means of supporting the target groups. ²⁵

Grant beneficiaries reported that the conceptual framework of the JCIV programme allowed them to develop their projects as planned. Of 61 respondents to the online survey, 59 said they had been able to reach the target groups. In addition, 86.8% of respondents to the online survey (53) reported feedback from their target group(s) confirming that the implemented project/activities were relevant to the problems or issues they faced. In follow-up interviews, most stakeholders said that they did not recall the specific messages of the target groups but that their projects were, in general, positively received. ²⁶

The overall conclusion that the JCIV programme and its priorities were relevant to grant beneficiaries was confirmed in stakeholder interviews. 10 out of 15 interviewees indicated that the conceptual framework of the programme (i.e. its objectives and the priorities set) had enabled them to develop their projects as originally envisaged. ²⁷

2.2COHERENCE AND COMPLEMENTARITY

Coherence relates to the extent to which the intervention logic of the programme is internally consistent, i.e. between objectives, inputs, activities and expected results, and to whether the intervention is consistent with interventions having similar objectives. Complementarity concerns the extent to which there is convergence with other policies and interventions, or overlap. Complementarity and coherence can be achieved by creating synergies, i.e. concerted actions, collaboration or interaction involving different actors/stakeholders mutually reinforcing shared goals. However, overlaps can be identified among the different programmes. Coherence and complementarity were analysed in terms of whether the programme was optimised in relation to other EU programmes and how the results of the implemented actions complement national policy initiatives/programmes and other European or international initiatives/programmes. It can be multi-dimensional in terms of (i) objectives and thematic areas; (ii) the priorities set in the calls for proposals and (iii) the actions funded.

Article 12 of the founding decision outlines the scope for complementarity with other EU financial programmes: the Criminal Justice Programme (JPEN), as part of the General Programme ‘Fundamental Rights and Justice’; Security and Safeguarding Liberties, composed of the two financial programmes, i.e. Prevention and Fight against Crime (ISEC) and Prevention, Preparedness and Consequence Management of Terrorism and other Security Related Risks (CIPS); Solidarity and Management of Migration Flows, composed of four funds, i.e. External Borders Fund (EBF), European Return Fund (RF), European Refugee Fund (ERF) and European Fund for the Integration of third-country (i.e. non-EU) nationals (EIF). In addition to the programmes referred to in the founding decision, further complementarity is possible with a number of other EU instruments, e.g. the FRC programme (in terms of certain objectives and thematic areas, the nature of the programme, and to some extent the target groups), and the Daphne programme (in terms of the nature of the programme and target groups). ²⁸

2.2.1Objectives and thematic areas

The programme achieved coherence and complementarity with programmes similar in objectives and themes by means of decisions made on the scope of the programmes at the programme design and call development stages. However, the analysis found both complementarity and overlap between the JCIV and the JPEN, FRC and (potentially) ISEC programmes, in terms of objectives and themes.

The JCIV and JPEN both aim to establish a European area of justice based on the principle of mutual recognition of court rulings, cooperation between judicial authorities in civil and criminal matters, and fostering trust. They are also intended to facilitate access to the courts and hence to facilitate the lives of Europeans, involving stakeholders and civil society in the debate on justice. There is scope for complementarity because they have similar objectives. The JCIV deals with civil and commercial matters, while the JPEN deals with criminal matters. But there is always a risk they will overlap, especially in general issues such as e-justice or training. Moreover, some specific subjects fall under civil justice in some Member States, and under criminal justice in others. Protection measures are one example: these are regulated by two separate instruments at EU level, a civil ²⁹ and a criminal one ³⁰ . The programmes were kept separate in 2007 because of their different legal bases, which called for different adoption procedures. However, this is no longer the case since the entry into force of the Treaty of Lisbon. The programmes are very similar in terms of objectives, management methods, and end beneficiaries and have in fact been combined into one funding instrument for the 2014-2020 period.

There is complementarity with the FRC programme’s objective of providing ‘training to legal professions and legal practitioners’ ³¹ which could be interpreted as similar to JCIV’s objective of ‘improving mutual knowledge of EU countries’ legal and judicial systems in civil matters for professionals and legal practitioners’.

The JCIV programme aims to ‘foster judicial cooperation in civil matters’. Article 3(3) of ISEC’s founding decision states that the programme does not deal with judicial cooperation. However, it may cover actions which aim at encouraging cooperation between judicial authorities and law enforcement authorities and therefore there is potential scope for complementarity and for overlap with the JCIV.

There is significant measure for complementarity between the JCIV programme and other EU funding programmes and little evidence of overlap. In particular the relatively narrowly defined scope of the JCIV programme (focusing on a specific thematic area, i.e. promoting access to justice and fostering judicial cooperation in civil matters, for particular types of beneficiaries, i.e. legal practitioners and national authorities) helped minimise overlap and duplication with other EU programmes. Particularly on cross-cutting issues such as e-justice and training there is a need for complementarity (also in order to avoid overlap). ³²

2.2.2Priorities set in the calls for proposals

When preparing annual work programmes, DG Justice also sought to increase complementarity and coherence between the DG Justice programmes by organising meetings between the programme management and policy units. At the same time, it participated in inter-service consultations and inter-service groups to discuss complementarity and possible overlaps in programming with other DGs. The Commission presented the annual work programme and its priorities to the Programme Committee for Member States to identify any complementarity and coherence issues with other EU or national initiatives. ³³

2.2.3Actions funded

A certain measure for complementarity was also found in a small number of projects implemented under the JCIV. This included a few cases of potential overlap with other EU programmes. Among the most important ³⁴ :

• The JCIV project: ‘Understanding Justice’ is similar to JPEN in that it aims to adapt the existing corpus of knowledge and practical recommendations for Legal Interpreting and Translation (LIT) in the Criminal Justice domain to the delivery of LIT in the Civil Justice domain, and to create and disseminate knowledge to judicial staff and practitioners, interpreters practising in the Criminal Justice domain and the public about best practices in interpreting and translation in civil justice proceedings.

• The JPEN project: ‘European Training Platform for lawyers’, created under the e-justice priority, aimed at creating a European Training Platform for both civil and criminal justice practitioners, although funded under the JPEN programme.

• The JCIV project: ‘Conference on Justice in the EU — from the Citizen’s Perspective’ held a conference which contributed to deeper understanding of the issue of individuals’ access to civil and criminal justice in the EU and improved networking among legal practitioners. The specific objectives of the JPEN programme include improving mutual knowledge of Member States’ legal and judicial systems in criminal matters, promoting and strengthening networking and improving information on legal systems in the Member States and access to justice.

• The JCIV project: ‘Litigious Love: Same-sex couples and mediation in the European Union’ helps to combat homophobia and targets same-sex couples in the EU. These objectives and target groups are similar to the FRC programme.

The online survey found various synergies with other grants funded by other EU, national or regional programmes and within the JCIV programme. In summary, nine respondents to the online survey reported synergies with other JCIV projects, seven reported synergies with other EU programmes, and six reported synergies with other national or regional programmes with similar objectives. ³⁵

2.3EFFECTIVENESS

The effectiveness of a programme refers to the extent to which the programme has been successful in achieving its objectives, and the extent to which JCIV projects were successful in achieving their own project objectives.

2.3.1Programme achievements

It is difficult to assess the effectiveness of the programme because of the limited information available on outcomes and impacts. ³⁶ The Commission's monitoring and evaluation system was rather output-oriented than results-oriented which explains some of the deficiencies of the present evaluation report.

Neither the action grant final report template nor the operating grant final report template has a section specifically requiring a description of outcomes. Moreover, the final reports reviewed tended not to differentiate between the projects’ outputs and outcomes or impact. This might be because neither the final report template nor the guidelines providing instructions for completing the report provided a clear explanation of the difference between outputs and outcomes or impact. ³⁷

Out of 146 action grants and operating grants implemented during the programme period, 107 were finished at the time information was made available for this evaluation. However, only 94 had a final report ready. A review of the final reports for these projects found 54 action grants and six operating grants with some evidence of useful outcomes. ³⁸

2.3.2Project achievements

Overall, the actions funded made a contribution to the programme objectives. The majority of the projects were able to achieve the planned results on time or to reach the expected target group. No obstacles seriously influencing the implementation of the project’s results or objectives were identified apart from those linked to the economic crisis. With regard to unexpected effects of the projects, the evidence gathered showed that the majority of the projects had reached unexpected positive results and almost no unexpected negative effects.

Over half of all finalised AGs and OGs (69 out of the 94 grants for which final reports were available) implemented all the activities as planned. In some cases, achievement of this entailed changes in the staff or required searching for additional funding. Out of 37 respondents to the online survey, 16 had to replace a member of staff in the course of the project/activities, while 10 respondents replaced more than one member of staff in the course of the project/activities and 7 respondents had to search for additional funding to implement the activities planned. Nonetheless, these fluctuations had a minor impact on the development of the funded actions. In fact, 85% of the 33 respondents to the online survey indicated that these changes had no impact on the outcomes of the project/activities. For example in the 2013 AG project ‘Business & Human Rights challenges for cross border litigation in the European Union’ although more than one member of staff had to be replaced, all of the project’s objectives were achieved.

Out of a total of 61 respondents, the majority stated that they were able to achieve all (56%), or most (26%), of the planned results on time and out of 52 respondents, 94% were able to reach the expected target groups. Moreover, out of 52 respondents, 58% reported that their project made a significant difference to the thematic area they were working in.

According to the evidence gathered through the online survey and follow-up interviews, it seems that the main internal factors positively influencing the implementation of the project were good relations among partners and good cooperation with the Commission. No obstacles seriously influencing the implementation of the project results or objectives were identified. However, some changes were introduced to the initial design of some projects to better achieve the planned results or to adapt to changing circumstances.

In relation to changes during the project/activities, out of 33 respondents, 55% reported that this change(s) had no impact on the implementation of the project/activities, and 42% reported that the above-mentioned change(s) had a minor impact. Only 3% reported that the change(s) had a major impact. Regarding the effect of these changes on outcomes, out of 33 respondents, 85% reported that the change(s) had no impact on the outcomes of the project/activities; 12% believed that there had been a minor impact and only 3% stated that the change(s) had a major impact on the outcomes of the project/activities. During follow-up interviews, project partners also cited as positive factors that contributed to effective implementation of projects and activities, the development of good relationships between partners and the cooperation with the Commission.

All 16 operating grants were completed between 2007 and 2013. The activities carried out mainly related to awareness-raising, mutual learning and support for key actors. For example, the 2009 operating grant for Save the Children Italy helped its legal unit carry out activities to foster judicial cooperation in civil matters and improve information on the legal systems and access to justice in Member States. These included producing and sending an online newsletter to 1000 recipients in 13 Member States, holding two meetings for judges and central authorities and four internal training meetings for Save the Children legal experts. It also helped with setting up the EU section of the International Association of Youth and Family Judges and Magistrates (IAYFJM). ³⁹

Under the public procurement component of the JCIV programme, 247 contracts were awarded. The information provided for the evaluation includes the total amount of the contract, the name of the lead contractor and whether the contract was for carrying out studies, events or IT-related tasks. The information shows ⁴⁰ that public procurement contracts were used for developing, implementing and disseminating EU policy and legislation, and to ensure that project results were sustainable.

Public procurement contracts are used to fund the European Day of Civil Justice, which is celebrated yearly on 25 October. These central events, co-organised by the Council of Europe and the European Commission, aim to bring justice closer to citizens and to inform them of their rights by staging mock cases and holding information sessions open to students, legal professionals and the general public, and ‘open days’ at courts. Awareness-raising activities (e.g. in 2012 a campaign on consumer rights) and dissemination activities (e.g. producing guides in various language versions, e.g. in 2010 Guide du citoyen, in 2011 Practice Guide for the Application of the Regulation on the European Order for Payment and in 2012 'Procédure européenne d’injonction de payer' are also funded through public procurement contracts. The European Judicial Atlas in Civil Matters should also be mentioned, and the contracts funded for the maintenance of the e-Justice Portal.

2.4SUSTAINABILITY, TRANSFERABILITY AND INNOVATION

In terms of sustainability, it was analysed, whether the results, outcomes and impacts achieved by the projects are sustainable beyond the project funding period.

Three levels of sustainability were identified in the evaluation:

• short-term sustainability, which is achieved mainly through dissemination of projects’ results;

• medium-term sustainability, which includes continuation of project results and/or partnerships; and

• longer-term sustainability, achieved mainly by transferring projects’ results to other contexts, organisations or Member States with little or no additional funding.

2.4.1Continuation of project activities and outputs and dissemination

Development of electronic tools has been a priority for JCIV-funded projects since the 2008 Communication and e-Justice action plan. This might explain why survey respondents indicated that they had continued or would continue to maintain hardware and/or software.

In follow-up interviews, 7 out of 15 interviewees indicated that electronic devices and online portals resulting from the projects were long-lasting outputs, although 2 out of 7 also pointed to difficulties in maintaining and updating them. Commission officials confirmed that updates (and maintenance) of the outputs (e.g. websites, studies) are a major concern and this may influence project selection. The question is whether to continue funding useful projects with good quality outputs which may not be sustainable in the long term, if the information is not regularly updated. ⁴¹

The evidence gathered ⁴² through the consultation with grant beneficiaries for this evaluation also suggests that projects heavily relied on EU funds to guarantee outputs. EU funding also contributed significantly to the ability of organisations to continue activities that were previously implemented.

In fact, more than half of the participants in the survey, i.e. 27 out of 52 survey participants, indicated that further funding was needed to guarantee financial sustainability of the project/activities

In some cases, the grant beneficiaries reported in the follow-up interviews that specific elements of their projects required further funding to ensure sustainable results — mostly to further disseminate project outputs and to transfer parts of the project. These specific elements are:

• maintenance of websites or electronic tools, i.e. electronic registry or ‘clearing houses’ such as an online mediation portal;

• follow-up activities to ensure the continuation of project results and extension of these results to a wider group of beneficiaries, e.g. conferences, events, and response to requests and questions regarding the project results;

• production of follow-up materials such as a manuals;

• transferability of results achieved within one Member State to other Member States, e.g. comparative analysis and exchange of best practices;

• training activities — in particular, adapting activities to other contexts.

Out of the 52 survey respondents, only 13 stated that further funding had already been secured from other EU funds. However, no further information on the type of additional funding was provided. In some cases, the organisation would fund follow-up activities itself, e.g. the Max Planck Institute, Luxembourg. ⁴³

2.4.2Continuation of partnerships after the project’s completion

The final reports of JCIV-funded actions suggest that the continuation rate of partnerships was remarkably low. Of the 146 mapped on the basis of the information available in the project documentation, only 10 reported continuing with their established partnerships — fully in 6 cases and partially in 4 cases. The continuation rate could be low because, as indicated in the mid-term evaluation report, the calls were very technical (focusing on highly technical legal subjects). This may have affected partnership options at the end of the projects. ⁴⁴

However, the online survey showed different results regarding whether partnerships continued after project completion. Of the 52 respondents, 69% indicated that the partnership continued and only 6% indicated that the partnership did not continue. The data could be contradictory because the mapping was carried out on the basis of the final reports (which might have provided incomplete information on this topic) while the surveys directly gathered the views of the project managers. The survey data is likely to be more representative of the actual situation with respect to the continuation of the partnerships entered into in a JCIV-funded project. ⁴⁵

2.4.3Potential sustainability and transferability of outputs

The final reports provided little information on the sustainability of projects’ outputs. Disseminating the outputs and results to a wide audience can increase their sustainability. Many of the outputs were already designed to be used at EU level (such as the EJTN’s judicial training was designed for participants from multiple Member States and the e-Justice portal for use by audiences in all Member States). Some outputs that were not designed for cross-EU application were otherwise designed to be transferable to other Member States. Many of the project partners interviewed for the evaluation demonstrated that they had designed their projects around clear products, or outputs, many of which would be or could be transferred to other Member States. A total of 70 out of 89 respondents to the online survey also expressed the view that all or some outputs could be used without any changes in more than one country and an additional 22 reported that the transfer could be done with some minor changes in more than one country. More than two thirds (61 respondents) also reported that the results of the project/activities could inform policy activities in one or more other countries. ⁴⁶

There was evidence that at least some actions implemented under the JCIV continued once EU funding ended. Out of 94 projects (action grants and operating grants) for which final reports were available, 45 provided some evidence of continuation of the project results. The reports on the other 49 actions funded did not provide information on this issue. The availability of information, therefore, affects the possibility of evaluating the sustainability of JCIV projects. ⁴⁷

Grant beneficiaries considered that many projects had achieved sustainable results. Out of 52 respondents, 30 (58%) reported that their projects made a significant difference to the thematic area they were working in; 18 respondents (35%) did not know, and 4 (8%) believed that their project did not make a significant difference. ⁴⁸

In some cases, problems with the transferability of results were identified — these are linked to the specific focuses of some of the actions implemented. This was the case in the project ‘European experience in mediation and its implementation in the Czech legal environment’ aimed at training Czech lawyers and judicial staff in mediation. The project ‘The application of Rome I and II Regulations and EC 4/2009 by legal practitioners in southern Europe – EUROME’, concerned Regulations focused on the law of the sea, and thus would not be implementable in countries with no relevance to this field. However, in both cases, the models developed could be used for other purposes.

The elements of projects that were transferred included:

• methodological tools created for the projects, e.g. methodological assessments, and activity models, e.g. to carry out campaigns, events, and activities with other target groups;

• websites — if content is translated, to be used in other countries;

• toolkits and project tools, e.g. reports, guidelines and manuals; and

• training courses and materials. ⁴⁹

2.4.4Innovation

Innovation is the development and implementation of new ideas (products, services and models) to meet needs and to create new relationships or collaboration. Innovation is about designing new responses to pressing (social) demands.

The evidence gathered showed that most projects were innovative in terms of activities or practices and the methods used. It also confirmed that the projects could be considered as leading the way forward in their field. Grant applicants were required to document the innovative aspects of their project. 140 projects showed some evidence of innovation. This is confirmed by the replies to the online survey. In fact, 90% of the 52 online survey respondents considered their projects/activities to be innovative.

Innovative practices and activities included training, ground-breaking research in an area never studied before, the creation of an online platform, and practices actively involving target groups, e.g. community discussions, the use of role models, and mapping of community resources. In terms of innovative methods, the research highlighted the use of tailor-made training methods, methods on how to involve target groups, e.g. businesses, judicial staff and mediators, interinstitutional cooperation models, and peer education methodology. ⁵⁰

Most respondents to the online survey (62% out of 52) also affirmed that the project could be considered as leading the way forward in their policy area. ⁵¹

A number of interviewees reported that stakeholders at local and European levels, e.g. professionals, institutions and social groups, were interested in the outputs of their projects — in particular, regarding the innovative aspects of implemented actions ⁵² , such as:

• devising instruments and methodologies which could be used directly by target groups to enhance their awareness and level of protection (for example, Lappeenranta University of Technology is creating an online ‘caseflow management’ platform offering practitioners information and best practices to avoid delays and inefficiencies in court management that could potentially undermine the functioning of the courts);

• addressing a field that has not been addressed before, e.g. the University of Pècs with its project ‘Improving the effectiveness of cross-border enforcement in the EU’, or the Romanian Ministry of Justice’s project to boost judicial cooperation on parental responsibility.

2.5EFFICIENCY AND SCOPE FOR SIMPLIFICATION

2.5.1Money spent in comparison with the impacts achieved

Most of the financial support within the JCIV programme’s budget was planned for public procurement contracts and action grants (see Table 1 2 Planned budget breakdown for the JCIV (2007-2013) ). The remainder of the budget was allocated to operating grants. High rates were allocated to public procurement contracts which were used extensively under the JCIV programme to help in developing and implementing EU policy and legislation, publicising EU policy and legislation and ensuring the sustainability of project results. To that end, a large amount of money was spent on IT tools and maintenance, particularly for the e-Justice portal, in addition to other tools, such as the European Judicial Atlas in Civil Matters and the JURE database. ⁵³

2.5.2Demand for action grants

There was a high demand for project funding, in terms of applications versus selected projects. In all, 385 applications for JCIV action grants were submitted between 2007 and 2013, and applications for the first calls had a higher success rate than those submitted for later calls. The 2007 and 2008 calls had a success rate of 61% of 44 and 31 applications respectively, but this proportion dropped to 20% for the 2011-2012 call (which covered two budget years), when applications reached a peak of 127, and to 29% for the 2013 call (with 96 applications). At the start of the programme the Commission did not receive the expected number of applications. However, the numbers rose sharply as the programme gained more visibility. The number of applications for action grants grew from an average of 40.5 in the period 2007-2010 to 127 in 2011-2012 and 96 in 2013. ⁵⁴

2.5.3Demand for operating grants

With a total of 50 operating grant applications (an average of 7 per year over the period, except for the 2010 call when a peak of 15 applications was reached), success rates were higher in 2008 and 2009 (50% and 43% respectively), dropping to 20% in 2010 and then rising to 29% and 38% in 2012 and 2013. ⁵⁵

2.5.4Use of public procurement

Some 36.2% of procurement expenditure was committed on studies; over 33.4% on IT services, and 26.8% on events. ⁵⁶ In addition to the Commission initiatives, JCIV procurement covers a budget for the management of the membership of the community in the Hague Conference on private international law and another allocated, according to the annual work programme, to the European Judicial Network in Civil and Commercial Matters, set up by Council Decision No 2001/470/CE of 28 May 2001 establishing a European Judicial Network in Civil and Commercial Matters. The former had little impact on the budget, while the latter was a commitment of EUR 17.8 million. This budget was planned for specific activities during the previous year and it was, overall, entirely committed and paid. ⁵⁷

2.5.5Overall implementation

As noted above, the initial total budget planned for the implementation of the JCIV in the programming period was EUR 103.7 million. Given that not all actions have been fully completed, the total amount paid is not yet known. The average underspend to date is 25%.

With these amounts, the JCIV funded 130 action grants, 16 operating grants and 247 procurement actions. For the procurement activities, it appears that all those completed to date were fully paid, which would also suggest efficient use of financial resources. ⁵⁸

2.5.6Scope for simplification

The evaluation of management efficiency and the scope for simplification is closely linked to the other findings on efficiency under this section and based on an assessment of the implementation process, e.g. whether it is overly complex (including unnecessary complicated administrative rules), or the reporting requirements too burdensome.

Based on the data collected to date, the Commission’s management of the JCIV was, overall, perceived as efficient in terms of the requirements imposed on applicants and grant beneficiaries and in view of the support received. However, some organisations experienced difficulties with the application and implementation process. For example, stakeholders noted complex procedure for applying in the calls for proposals and difficulties encountered with the technical and IT system, which has led to some organisations refraining from responding to the calls for proposals. There is some scope for simplifying the procedures for applicants and beneficiaries including: improving technical/IT systems; simplifying accounting procedures and financial reporting; improving and speeding up the feedback process; paying institutions for their work. Reporting arrangements were overall considered appropriate but sometimes cumbersome, particularly with regard to financial reporting. The Commission’s monitoring arrangements were overall perceived as useful, however some respondents experienced difficulties.

The Commission's monitoring and evaluation system was rather output-oriented than results-oriented which explains some of the deficiencies of the present evaluation report.

2.6EUROPEAN ADDED VALUE

EU added value refers to the extent to which the EU nature of the programme brings value to its stakeholders and the extent to which the EU has a comparative advantage over national and international actors working in the area. First, the EU nature of the programme and its geographical coverage is discussed. The EU added value is then analysed in terms of value to the EU and to beneficiaries. However, it has not been possible to identify and measure EU added value in terms of impacts.

In general, funding under programmes managed by DG Justice and covered by this evaluation was only available to activities with clear European ‘added value’. This means that activities must transcend national or local interests to become truly ‘European’ with European relevance, coverage and impact.

The JCIV programme’s added value mainly relates to its transnational dimension within the EU. The programme was established in order to develop an area of freedom, security and justice, in which the free movement of persons is ensured. Member States cannot effectively address this by themselves. Therefore the programme was implemented in order to require a European response, including the exchange of information at EU level and ensuring the dissemination of good practice throughout the EU. Cooperation between EU Member States is therefore crucial for the implementation of the programme.

For action grants, Article 4 of the founding decision stipulates that the JCIV funds specific transnational projects involving at least two Member States or at least one Member State and one other State, which may be either an acceding country or a candidate country. The annual work programmes and call for proposals require applicants to form transnational partnerships ‘of eligible organisations, i.e. an applicant plus a minimum of one partner, composed of organisations from at least two different eligible countries, i.e. from at least two different EU Member States’.

For operating grants, organisations had to carry out activities with a European dimension involving at least 10 EU Member States. Article 9(5) of the founding decision also states that one of the evaluation criteria for selecting operating grant applications is the ‘geographical impact of the activities carried out’.

According to Article 5 of the founding decision, acceding countries, candidate countries and Western Balkan countries included in the stabilisation and association process may participate in the actions of the programme. Furthermore, legal practitioners from Denmark, from candidate countries not participating in the programme, where this would contribute to their preparation for accession, or from other non-EU countries not participating in the programme, where this serves the aim of the projects, may participate as associate partners but are not permitted to submit projects or be co-applicants (co-beneficiaries). The themes covered by JCIV, as mentioned in the annual work programmes and calls for proposals, have strongly focused on EU matters and transnational cooperation. Chief among these, the 2010 annual work programme refers to the area of European e-Justice as a general priority, and in particular support for developing best practice through transnational pilot projects, which had to demonstrate how they have contributed to improving interoperability at EU level.

2.6.1Geographical coverage of the projects funded

From the evaluation, it appears that most JCIV grants went to a select group of Member States (Belgium, Germany, France and Italy). Other Member States were involved to only a limited extent or were not involved at all.

As the JCIV programme focused on improving judicial cooperation and mutual trust, it was crucial that the programme involved partners and final beneficiaries from different EU Member States. Most of the participant organisations were based in Belgium, Germany and Spain, with most of these organisations being partner organisations. Denmark did not participate in the JCIV programme.

Organisations from all Member States except Portugal, Malta, Cyprus, Croatia, Bulgaria, Slovakia and Denmark participated as lead organisations in the projects of the JCIV programme. The highest number of lead organisations was from Belgium (21% or 31 organisations), followed by Germany (12% or 18 organisations) and Spain (11% or 16 organisations). Belgium hosts a high share of EU-level networks, platforms and forums, which explains the high number of lead organisations registered in Belgium. The highest number of partner organisations was from Italy (10% or 57 organisation) followed by Spain (10% or 55 organisations). However, partner organisations were spread more evenly across different Member States.

21% of JCIV funding was committed to projects where the lead organisation was from Belgium. Belgium was followed by Germany (12%) and France (11%). This funding distribution closely follows the distribution of the number of projects led by Member States (see Figure 2 1 (left)). However, the funding map does not show the spread of committed funding among project partners. The Figure 2 1 assumes that all of the committed funding was allocated to the country of the lead organisation. This was not the case in reality (projects were transnational and project partners also received part of the funding), so the Figure 2 1 should be interpreted with caution.

Figure 21 Allocation of JCIV committed funding by lead organisation (left) and by lead organisation per capita (right)

Figure 2 2 further highlights the partnership structure of the top 3 Member States with the highest number of lead organisations (Belgium, Germany and Spain). Figures show that Belgian lead organisations commonly work with Belgian, French and German partners, and with Spanish and Dutch partners. German lead organisations most frequently partnered with Austrian organisations (eight projects) followed by Czech, German and Italian organisations. Spanish lead organisations mostly partnered with the Portuguese, Italian and Spanish organisations.

Figure 22 Partnership structure for the top three Member States of lead organisation

The JCIV programme did not cover all Member States equally. The highest number of applications was submitted by Italian organisations (76 out of 385) followed by organisations from Belgium (69) and Spain (55). These top three applicant Member States submitted 52% of all applications for funding from the programme. Croatia, Malta, Cyprus and Denmark did not apply for JCIV funding (see Figure 2 3 ).

Figure 23 Total number of applications in JCIV per Member State (right) and the success rate by Member State (left)

Luxembourg, Finland, Czech Republic, France and Sweden showed the highest success rates of the applications submitted (more than 50% of the applications were successful). Portugal, Bulgaria and Slovakia did not obtain funding from the JCIV programme.

2.6.2Added value for the EU and for grant beneficiaries

The follow-up interviews and online survey showed that the transnational partnership brought specific benefits to the beneficiary organisation, which no other funding source would have enabled on the same scale ⁵⁹ . The partnership approach has resulted in an increased knowledge base of participating organisations. Out of 59 respondents to the online survey, 45 confirmed that the partnership resulted in more knowledge/expertise in the topic area (how much their knowledge has increased was not measured), in networking with (more) international partners (41 respondents) and in more knowledge on policy and practice in other countries (38 respondents). To a lesser extent, it also contributed to more knowledge of relevant EU legislation and EU policies. ⁶⁰

The transnational partnership also further extended the scope for dissemination of results. The survey results, together with stakeholder consultations, indicate that good working relations between partners enabled organisations to reach all target groups, including those difficult to reach, and had ensured wide dissemination of the results. Engagement of all partners in the project, clear task allocation, and agreement on how to implement activities, had helped them obtain these results. However, despite overall satisfaction with the partnerships, 26% of the online survey respondents strongly agreed, and 35% partially agreed, with the statement that it would have been useful to involve partners from other countries. Follow-up interviews also confirm the specific benefits of transnational partnerships, i.e. a larger knowledge base and greater scope for dissemination, and good working relations between partners underpinning these specific benefits.

In addition to added value for grant recipients, the survey indicates that the projects added value by contributing to achieving the EU objectives. Most survey respondents reported that the implemented project/activities made a significant/major contribution to improving cross-border cooperation and the elaboration and dissemination of best practices. Among these, analytical activities mainly contributed to other objectives (40%), mutual learning mainly contributed to developing mutual trust among countries (28%) and awareness-raising mainly contributed to the elaboration and dissemination of best practices (25%).

However, most respondents reported that the implemented project/activities only made a minor or no contribution to creating practical tools and solutions that address cross-border or Union-wide challenges and to making a wider public aware of rights deriving from EU law.

The EU added value is particularly evident in the responses of grant beneficiaries concerning how important EU funding was to achieving the objectives. 87% of survey respondents stated that the projects/activities would not have been implemented without EU funding. It can therefore be assumed that most of the project/activities carried out under the JCIV programme would not have been implemented had the programme not existed. ⁶¹

3CONCLUSIONS

Relevance of the programme ⁶²

• The programme objectives were largely specific, attainable and realistic. However, they were neither measurable, nor time-bound (although bound by programme and project duration).

• Overall the priorities and funded actions can be considered relevant to the objectives of the programme. However, the process put in place to select the programme’s priorities left little room for manoeuvre, once the priorities had been set (although in principle a project which did not reflect any priority could receive a grant under other award criteria). This meant that in some cases, if a priority changed during the implementation of a project, its results were less useful. Conversely, with regard to training, for example, needs could arise that did not fall under the set priorities. They could not then be funded under the programme because they were not taken into consideration in a specific call.

• The JCIV programme was well designed to support the practical application and implementation of EU policy and legislation, and to contribute to policy and legislative development.

• Overall, grant beneficiaries found that the calls and selected actions identified, and responded to, the needs of the target groups, although the extent to which they adequately identified these needs varied, with some undertaking elaborate needs assessments and others adopting a less rigid approach. However, the robustness of the method used for the needs assessments carried out by the projects could not be assessed by this evaluation. Needs assessments are not a mandatory requirement for the grant beneficiaries.

Coherence and complementarity

• Overall, the JCIV achieved a certain degree of complementarity with other EU programmes in terms of certain objectives and thematic areas, the nature of the programme and the target groups, especially with the JPEN and FRC. At the same time, however, there is a risk of overlap with these programmes.

• Synergies were established at EU and national levels and within the programme itself.

• The JCIV programme was consistent with policy developments at EU and national levels and was able to support policy developments and legislation in civil justice through the actions funded and through public procurement contracts.

Effectiveness

• Overall, the actions funded made a contribution to the programme objectives. However, it is difficult to assess the effectiveness of the programme because of the limited information on outcomes and impacts provided in the projects final reports and the interviews. The Commission's monitoring and evaluation system was rather output-oriented than results-oriented which explains some of the deficiencies of the present evaluation report.

• Most of the projects were able to achieve their planned results on time and reach the expected target group. No obstacles seriously influencing the implementation of the project’s results or objectives were identified.

• The JCIV has contributed to the implementation and development of EU policy and legislation, via grants and public procurement contracts. However, the lengthy project cycle could nullify the effectiveness of the finalised actions where policy priorities changed.

Sustainability

• Regarding short-term sustainability (i.e. through dissemination of project results), the Commission’s efforts to disseminate (and monitor) project results have been limited, overall. The Commission mostly relied on its beneficiaries to disseminate the outputs directly to their target groups, and its own human resources were dedicated mostly to the financial management. Dissemination to a wider public is identified as key element in giving the programme greater impact. Dissemination of JCIV results was limited in effectiveness by the lack of a specific channel for disseminating them. Grant beneficiaries’ dissemination appeared to be more efficient, as most had established clear plans for disseminating the results of their project/activities.

• Overall, the JCIV programme generated sustainable results in the medium (i.e. continuation of project results and/or partnerships) and long term (i.e. through the transfer of projects’ results to other contexts, organisations and Member States with little or no additional funding). Regarding the continuation of partnerships, the evidence collected is somewhat contradictory: while the JCIV final reports suggested that this was low, the online survey showed a different result, with most respondents indicating that the partnership had continued. This could be due to partnerships being ‘reconstituted’ only after a while, when the final report had already been submitted.

• Most projects were considered innovative, particularly in targeting groups or tackling issues that had not been researched or addressed before, or through the development of new methodologies, approaches or tools.

• More than half of survey respondents indicated that further funding was needed to guarantee financial sustainability of the project/activities.

Efficiency

• Overall, funding provided to action grants, operating grants and procurement contracts appears to have been sufficient to support the achievement of the JCIV’s general objectives.

• The financial resources have been used efficiently overall, judging by the comparison of inputs and outputs between projects’ budgets, and the total outputs and results produced. In the first years of implementation, absorption rates were relatively low, pointing to a lack of financial and administrative capacity of the potential beneficiaries, but this increased to an acceptable level after two years.

• The Commission’s management of the JCIV was perceived as efficient in terms of the requirements imposed on applicants and grant beneficiaries and of the support received. However, some organisations found the application and implementation process difficult. There is some scope for simplifying procedures for applicants and beneficiaries (e.g. improving technical/IT systems or simplifying accounting procedures and financial reporting). Reporting arrangements were considered appropriate overall, but sometimes cumbersome, particularly with regard to financial reporting. The Commission’s monitoring arrangements were considered useful overall.

EU added value

• The JCIV programme had a strong transnational dimension. The transnational partnerships resulted in specific benefits for the organisations involved in implementing JCIV activities, e.g. an increased knowledge base of participating organisations, networking consisting of (more) international partners and improved knowledge of policy and practice in other countries.

• The geographical coverage of action grants and operating grants was uneven, with a few Member States overly represented (Belgium, Germany, France and Italy) and others involved little or not at all.

Key recommendations

• Better define the priorities: the Commission should invest more time and human resources in the process of setting priorities in order to ensure that the priorities can be adequately achieved within an earmarked budget.

• Realistic assessments of project risks and better risk mitigation strategies: the Commission should better monitor risks throughout the project duration, for example by asking for brief progress reports that identify any potential risks as they arise during the implementation of the project.

• Increase focus on assessment of impacts at all levels and not merely on outputs, as regards monitoring and evaluation. This goes hand in hand with the need to collect, analyse and use objective and independent evidence in order to perform project and programme evaluations. Increase focus on needs assessment that each project aims to address.

• Explore ways of enhancing the uptake of project outputs, results and best practices by other organisations, including in other Member States, including more resources for translations, communication and dissemination.

• Sharpen the programme's intervention logic; further to the scope of the programme and its general and specific objectives and priorities, types of action and types of intervention and implementing measures, the Commission shall seek to sharpen the intervention logic ⁶³ , and make the relations between the rationale, objectives, inputs, outputs, beneficiaries, expected outcomes and impacts articulate, precise and concrete in any future continuation of the programme.

(1)

Article 16(3)(d) of Decision No 1149/2007/EC of 25 September 2007 establishing for the period 2007-2013 the Specific Programme Civil Justice as part of the General Programme Fundamental Rights and Justice.

(2)

The ex-post evaluation report by external evaluator is published here: Main report: http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jciv_programme_evaluation_final_report.pdf , Annexes: http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jciv_annex_4_quantitative_analysis.pdf

Mid-term evaluation report by external evaluator is published here: http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2011:0351:FIN:EN:PDF

(3)

These two sources are rather subjective sources of evidence, and cannot be taken for independent evidence of performance.

(4)

Article 8 of Decision No 1149/2007/EC of 25 September 2007 establishing for the period 2007-2013 the Specific Programme Civil Justice as part of the General Programme Fundamental Rights and Justice.

(5)

Ibid.

(6)

Ibid.

(7)

Article 13 of Decision No 1149/2007/EC of 25 September 2007 establishing for the period 2007-2013 the Specific Programme Civil Justice as part of the General Programme Fundamental Rights and Justice.

(8)

Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Civil Justice (JCIV), ICF, 28 July 2015, p. 38 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jciv_programme_evaluation_final_report.pdf

(9)

Budget allocated to action grants for ‘specific trans-national projects of Community interest presented by an authority or any other body of a Member State, an international or non-governmental organisation, and involving in any case at least two Member States or at least one Member State and one other State which may be either an acceding country or a candidate country’ (Article 4(b) of the founding Decision).

(10)

Budget allocated, according to the annual work programme, to action grants for specific projects to help implement European competition rules. These are projects to promote judicial cooperation between, and training of, national judges in the context of the enforcement of the European competition rules, including Articles 101 and 102 of the TFEU and the State aid rules, with the aim of contributing to the development and implementation of European competition policy and Community cooperation measures in the field of competition to ensure the consistency of Community competition policy.

(11)

Budget allocated to operating grants for ‘the activities of non-governmental organisations or other entities pursuing an aim of general European interest in accordance with the general objectives of the Programme under the conditions set out in the annual work programmes’ (Article 4(c) of the founding Decision).

(12)

Budget allocated to ‘operating grants to co-finance expenditure associated with the permanent work programmes of the European Network of Councils for the Judiciary and the Network of the Presidents of the Supreme Judicial Courts of the European Union, insofar as it is incurred in pursuing an objective of general European interest by promoting exchanges of views and experience on matters concerning case-law and the organisation and functioning of the members of those networks in the performance of their judicial and/or advisory functions with regard to Community law’.

(13)

Budget allocated to the management of the membership of the community in the Hague conference on private international law.

(14)

Budget allocated, according to the annual work programme, for operation of the European Judicial Network in Civil and Commercial Matters, created by Council Decision No 2001/470/CE of 28 May 2001, establishing a European Judicial Network in Civil and Commercial Matters.

(15)

Budget allocated to ‘specific actions initiated by the Commission, such as studies and research, opinion polls and surveys, formulation of indicators and common methodologies, collection, development and dissemination of data and statistics, seminars, conferences and expert meetings, organisation of public campaigns and events, development and maintenance of websites, preparation and dissemination of information material, support for and management of networks of national experts, analytical, monitoring and evaluation activities’ (Article 4(a) of the founding Decision).

(16)

Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Civil Justice (JCIV), ICF, 28 July 2015, p. 3 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jciv_programme_evaluation_final_report.pdf

(17)

Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Civil Justice (JCIV) quantitative analysis, 28 July 2015, p. 10 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jciv_annex_4_quantitative_analysis.pdf

(18)

SMART objectives are defined as being specific, measurable, achievable, results-focused and time-bound.

(19)

Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Civil Justice (JCIV), ICF, 28 July 2015, p. 7 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jciv_programme_evaluation_final_report.pdf

(20)

European Commission communication Building trust in EU-wide justice — A new dimension to European judicial training, COM/2011/0551 final, 13.09.2011.

(21)

E.g. to the EJTN via the JPEN programme and to other organisations, such as the Academy of European Law (ERA) and the European Institute of Public Administration (EIPA), through operating grants from the Directorate-General for Education and Culture.

(22)

Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Civil Justice (JCIV), ICF, 28 July 2015, p. 7 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jciv_programme_evaluation_final_report.pdf

(23)

Regulation (EU) No 650/2012 on jurisdiction, applicable law, recognition and enforcement of decisions and acceptance and enforcement of authentic instruments in matters of succession and on the creation of a European Certificate of Succession.

(24)

Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Civil Justice (JCIV), ICF, 28 July 2015, p. 7 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jciv_programme_evaluation_final_report.pdf

(25)

Ibid., p. 12

(26)

Ibid., p. 13

(27)

Ibid., p. 12

(28)

Ibid., p. 14

(29)

Regulation (EU) n. 606/2013 on mutual recognition of protection measures in civil matters.

(30)

Directive 2011/99/EU on the European protection order.

(31)

European Commission website, Fundamental Rights and Citizenship Funding Programme, available at http://ec.europa.eu/justice/grants/programmes/fundamental-citizenship/index_en.htm .

(32)

Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Civil Justice (JCIV), ICF, 28 July 2015, pp. 14-15 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jciv_programme_evaluation_final_report.pdf

(33)

Ibid., p. 8

(34)

Ibid., p. 17

(35)

Ibid., p. 18

(36)

Ibid.

(37)

Ibid., p. 19

(38)

Ibid., p. 19

(39)

Ibid., p. 23

(40)

Ibid.

(41)

Ibid., p. 32

(42)

Ibid.

(43)

Ibid., p. 33

(44)

Ibid., p. 32

(45)

Ibid.

(46)

Ibid., p.27

(47)

Ibid., p. 31

(48)

Ibid., p. 27

(49)

Ibid., p. 34

(50)

Ibid.

(51)

Ibid.

(52)

Ibid.

(53)

Ibid., p. 37

(54)

Ibid., p. 39

(55)

Ibid.

(56)

Ibid.

(57)

Ibid.

(58)

Ibid.

(59)

87% of survey respondents to this question stated that the projects/activities would not have been implemented without EU funding.

(60)

Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Civil Justice (JCIV), ICF, 28 July 2015, p. 52 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jciv_programme_evaluation_final_report.pdf

(61)

Ibid., p. 53

(62)

Ibid., pp. 54-56

(63)

See for instance Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Civil Justice (JCIV), ICF, 28 July 2015, pp. 1-2 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jciv_programme_evaluation_final_report.pdf .