Legal provisions of COM(2017)115 - Ex post evaluation report on the Criminal Justice Programme (2007-2013)

Please note

This page contains a limited version of this dossier in the EU Monitor.

dossier COM(2017)115 - Ex post evaluation report on the Criminal Justice Programme (2007-2013).
document COM(2017)115 EN
date March  7, 2017
EUROPEAN COMMISSION

Brussels, 7.3.2017

COM(2017) 115 final

REPORT FROM THE COMMISSION TO THE COUNCIL AND THE EUROPEAN PARLIAMENT

Ex post evaluation report on the Criminal Justice Programme (2007-2013)


TABLE OF CONTENTS


1    INTRODUCTION    

1.1    OBJECTIVE OF THE REPORT    

1.2    METHODOLOGY AND SOURCES OF INFORMATION    

1.3    OVERVIEW OF THE PROGRAMME    

2    RESULTS OF THE EVALUATION    

2.1    RELEVANCE OF THE PROGRAMME    

2.2    COHERENCE AND COMPLEMENTARITY    

2.3    EFFECTIVENESS    

2.4    SUSTAINABILITY AND TRANSFERABILITY    

2.5    EFFICIENCY AND SCOPE FOR SIMPLIFICATION    

2.6    EUROPEAN ADDED VALUE    

3    CONCLUSIONS    


1INTRODUCTION

1.1OBJECTIVE OF THE REPORT

1 2 The Decision establishing the Criminal Justice Programme (JPEN) requires the Commission to present an ex post evaluation covering 2007-2013. This ex post evaluation was performed by an independent external evaluator assisted by Commission staff.


This report is based on that evaluation. The report is structured according to the main evaluation criteria and corresponding questions. These include relevance, coherence and complementarity, effectiveness, impact and sustainability, efficiency and scope for simplification, and European added value.

1.2METHODOLOGY AND SOURCES OF INFORMATION

This final evaluation of JPEN is based on the following:

• an extensive review of the available documentation for 284 action grants and 50 operating grants funded by the 2007-2013 Criminal Justice programme 2007-2013;

• a review of programme documentation, such as the founding decision, annual work programmes and call for proposals for both grants and public procurement contracts;

• a review of EU policy documents, founding decisions of related EU programmes, etc.;

• a quantitative analysis of 334 JPEN projects (action grants and operating grants);

• an analysis of 97 responses to the online survey from grant beneficiaries;

• the write-ups of 33 follow-up interviews with coordinators of projects/organisations receiving JPEN 2007-2013 grants who also responded to the online survey;

• interviews with Commission officials involved in the programme.

1.3OVERVIEW OF THE PROGRAMME

JPEN was established under the Council Decision No 2007/126/JHA of 12 February 2007 as part of the ‘Fundamental Rights and Justice’ General Programme. The programme was implemented for seven years from 2007-2013. It is a continuation of the framework programme on police and judicial cooperation in criminal matters (AGIS) (2003-2007).

JPEN addressed one of the four objectives of the Programme on Fundamental Rights and Justice, i.e. the need to promote judicial cooperation and to create a genuine area of justice in criminal matters in accordance with the principles of mutual recognition. It was conceived also in order to establish a computerised system of exchange of information of criminal data.

The five general objectives of JPEN, as listed in Article 2 of the Decision, are:

• to promote judicial cooperation with the aim of contributing to the creation of a genuine European area of justice in criminal matters based on mutual recognition and mutual confidence;

• to promote the compatibility of rules applicable in the Member States as may be necessary to improve judicial cooperation;

• to promote a reduction in existing legal obstacles to the proper functioning of judicial cooperation in order to strengthen the coordination of investigations and to increase compatibility of the existing judicial systems in the Member States with the European Union with a view to providing adequate follow-up to investigations carried out by law enforcement authorities of the Member States;

• to improve contacts, exchange of information and best practices between judicial and administrative authorities and the legal professions (lawyers and other professionals involved in the work of the judiciary), to foster the training of the members of the judiciary so as to enhance mutual trust;

• to improve further mutual trust with a view to ensuring protection of rights of victims and of the accused.

The specific objectives of JPEN, as listed in Article 3 of the Decision, are to:

• Foster judicial cooperation;

• Improve mutual knowledge of Member States' legal and judicial systems in criminal matters and to promote and strengthen networking, mutual cooperation, exchange and dissemination of information, experience and best practices;

• Ensure the sound implementation, the correct and concrete application and the evaluation of Union instruments in the areas of judicial cooperation in criminal matters;

• Improve information on legal systems in the Member States and access to justice;

• Promote training in Union and Community law for the judiciary, lawyers and other professionals involved in the work of the judiciary;

• Evaluate the general conditions necessary to develop mutual confidence;

• Develop and implement a computerised system of exchange of information on criminal records and to support studies to develop other types of exchange of information.


To achieve these objectives, the programme funded activities through three different types of financial support: action grants; operating grants; and procurement 3 .

Action grants were awarded for specific transnational projects of interest to the whole of the EU; national projects preparing for or complementing transnational projects or Union measures; and national projects contributing to developing innovative methods and/or technologies with a potential for transferability.

Operating grants were awarded either to NGOs or other bodies pursuing an aim of general European interest in the area of criminal justice; or to the European Judicial Training Network (EJTN), specifically to co-finance expenditure associated with their permanent work programme of judicial training. 4  

The Commission also used JPEN funding to procure specific actions of importance to the Commission. These included studies; research; and the creation and implementation of specific IT projects to facilitate the exchange and dissemination of information.

The 2010 annual work programme included specific action grants to co-finance European e-justice projects. In 2013, a specific call for proposals was launched to finance judicial training projects.

The programme was aimed at supporting the following target groups: legal practitioners; representatives of victims’ assistance services; other professionals involved in the work of the judiciary; national authorities; and the EU public in general.

The total planned budget for JPEN for January 2007 to December 2013 was EUR 196 million (see Table 1 1 Planned budgetary breakdown for JPEN 2007-2013 ). 5  

Table 11 Planned budgetary breakdown for JPEN 2007-2013

Action grantsOperating grants 6ProcurementTotal
Value (€)Value (€)Value (€)Value (€)
200722 000 0005 900 0001 300 00029 200 000
200818 800 0006 000 0005 000 00029 800 000
200914 900 0008 500 0007 000 00030 400 000
201017 000 0005 000 0004 000 00026 000 000
201114 500 0006 500 0006 000 00026 500 000
201213 450 0007 500 0006 000 00026 950 000
201315 210 0007 660 0004 640 00027 510 000
Total114 860 00049 560 00031 940 000196 360 000

Source: JPEN Programme — Annual work programmes (2007-2013)


Most of the financial support under JPEN’s budget was allocated to action grants (64 %). Operating grants to co-finance the annual work programme of NGOs and expenditure associated with the permanent work programme of the EJTN represented 20 % of the total programme budget. The remainder of the budget was allocated to public procurement contracts. 7

As described above, JPEN funding was made available through action grants and operating grants, framework partnership agreements and public procurement contracts. Table 1 2 summarises the number of different actions funded in each year of the programme implementation period. Calls for proposals for action grants and operating grants were split across years in 2011 and 2012.


Table 12 Number of actions funded per year

20072008200920102011201220122013TOTAL
Action grants5641324953-49280
Specific action grants--2-2--4
Operating grants57138-10750
TOTAL61484757551056334

Note: the table differs from the number of projects analysed for this evaluation since documentation was not available for all projects.


JPEN projects were primarily led by national authorities, such as ministries of justice/interior (22 % of all lead organisations); then national NGOs/platforms/networks (18 %); European networks/platforms/forums (17 %); and other education/training institutes (10 %).

This is consistent with what is stated in the founding decision under which organisations have access to the programme.

Action grants were awarded to organisations working together in partnership. The composition of the partnerships shows that the most common partners were national NGOs, including national platform and networks, universities and national authorities. Such partners represented 19 %, 16 % and 15 % respectively of the total figure. The composition of partnerships is not specifically mentioned in the founding decision. 8

Figure 1 1 below illustrates the main types of activities addressed by the JPEN action grants (left) and operating grants (right). Most action grant-funded projects focused on awareness raising, information and dissemination activities (22 %); then analytical activities (22 %); mutual learning, exchange of good practices, cooperation (21 %); and training activities (19 %).

The main activities implemented through operating grants were awareness raising, information and dissemination activities (18 %); then support to key actors (17 %); mutual learning, exchange of good practices, cooperation (15 %); and training activities (15 %).


Figure 11 JPEN action grants by main activity (left) and JPEN operating grants by main activity (right)

Public procurement contracts focused on three main activities such as studies; the organisation of events and meetings; and IT provision.

Public procurement was also used for maintenance of the e-Justice website. The procurement of services was also spent on expert services under the cooperation and verification mechanism for Romania and Bulgaria. 9

2RESULTS OF THE EVALUATION

2.1RELEVANCE OF THE PROGRAMME

The relevance of a programme is assessed in terms of the extent to which its actions logically address its objectives, the wider EU policy needs and the needs of the target audiences.

Analysis of the programme objectives and priorities shows that while these were largely specific, attainable and realistic, they were not measurable or (in most cases) time-bound. More detail was increasingly provided in the calls for proposals as to the calls’ expectations for funded projects (in relation to different priorities). The goals of most of the priority areas were aligned to EU objectives and policies, particularly in the last half of the programme. 10

Relevance to these objectives and to wider EU policy priorities was a key selection criterion in every action grant and operating grant call. An assessment of the operational objectives of the action grant and operating grant beneficiaries shows alignment between the projects’ operational objectives and the programme’s objectives. Furthermore, analysis of the action grants and operating grants demonstrates that all the action grant-funded actions and most of the operating grant-funded actions were clearly aligned with the programme’s priorities. 11

The kinds of activities covered in the programme were also highly relevant to the programme objectives and in line with Article 4 (‘eligible actions’) of Council Decision 2007/126/JHA on JPEN. A large number of actions selected for grants or in procurement from 2007 to 2013 fell into the following categories:

• training activities, mainly for the judiciary or legal practitioners;

• mutual learning/exchanges between authorities/organisations in Member States (e.g. through symposia, seminars and dissemination of good practices);

• development and/or installation of hardware/software to facilitate judicial proceedings and access to information about justice at EU and/or national level (e-Justice);

• support and advice services (including information and advice websites);

• research and analytical activities assessed the application of EU law in Member States or mapped national legislative frameworks in order to assess compatibility with EU legislation. Such activities also researched methods for implementing different forms of justice (e.g. restorative justice or alternatives to detention) and/or analysed obstacles to justice in the EU. Some included assessments of obstacles to the implementation of EU law and mutual confidence 12 .

In addition, grants were awarded to many of the key organisations involved in promoting a common European area of justice. For example, grants went to major European networks and NGOs working in training (e.g. EJTN); probation (the Confederation of European Probation — CEP); legal rights (Advice on Individual Rights in Europe — AIRE); victim support (Victim Support Europe — VSE); protection of suspects and defendants (Fair Trials); and restorative justice (European Forum for Restorative Justice — EFRJ). 13

As JPEN focused on improving judicial cooperation and mutual trust (and aimed to support the creation of an EU-wide judicial area), it was also crucial that the programme involved partners and end beneficiaries from different EU Member States. JPEN also included national actions (40 out of 284 action grants, see below, 2.6.2 Added value for the EU and for grant beneficiaries ). These were relevant actions to the programme.

2.1.1Priorities set in the calls for proposals and selected actions and their relevance to policy

The main legislative instruments in place and introduced during the time period in question comprised instruments that provided for the mutual recognition of Member State judicial decision; instruments that provided for alternatives to detention; instruments that support judicial cooperation in the provision of evidence to other Member States; instruments introducing provisions on procedural rights; and instruments to support victims and to ensure their rights. 

The JPEN annual priorities reflect changes in EU policy on criminal justice. 14

The Commission implemented and promoted different practical instruments to support the main legislative instruments. The aim was to improve judicial cooperation and the mutual recognition of decisions, and also increase the public’s access to justice. These practical instruments included 15 :

• the European e-Justice portal — an information website targeting the public, businesses, legal practitioners and the judiciary and gathering information on rights, services available and tools meeting their different needs;

• the European Criminal Records Information System (ECRIS) — a computerised system set up to achieve efficient exchange of information on criminal convictions between EU countries;

• the European Judicial Network (EJN) — a network to promote judicial cooperation in criminal matters between EU countries;

• judicial training.

From 2010, the priorities, as outlined in the calls for proposals and work programmes, became ‘SMARTer’ to the extent that they were more specific as to the expected content of the projects addressing the priority. They also became more responsive to policy and legislation. These improvements were the result of institutional changes to the Commission after the entry into force of the Lisbon Treaty, which saw greater powers being granted to the Commission in the area of justice and home affairs. In addition, in 2010, the Directorate-General for Justice, Freedom and Security (DG JLS) split into DG Home Affairs and DG Justice (with JPEN falling under the new DG Justice) with greater responsibility being given to both Directorates-General. 16

The actions funded through action grants and operating grants were mainly aimed at supporting the implementation of existing legislation. 17 Selected actions also indirectly supported new policy development to the extent that they facilitated the exchange of ideas and discussion of obstacles to justice/cooperation in the EU between ministries of justice and members of the judiciary/legal professionals in the EU. Out of the 334 grants, 16 mapped and produced policy recommendations. 18  

Because the priority areas of the calls for proposals were accompanied by a description of the expected content of the projects, this made them more specific and helped grant applicants to ensure that their projects were relevant to EU objectives. Furthermore, because the priority areas followed the programme objectives and EU policy objectives so closely, it was easy for the selected actions to target these objectives as well.

Almost all the organisations producing these outputs were key European platforms or NGOs that have a leading role in the monitoring of specific aspects of criminal justice in the EU, namely the Confederation of European Probation (CEP); the Council of Bars and Law Societies of Europe (CCBE); the European Organisation of Prison and Correctional Services (EuroPris); Fair Trials International (FTI); the International Juvenile Justice Observatory; VSE; and the Irish Council for Civil Liberties.

Since the above bodies are networks with an overview of the EU and well-established expertise in these areas, this makes them well placed to provide policy recommendations.

Services procured using JPEN funding were also relevant to the programme and wider EU objectives, as they focused mainly on developing the e-Justice portal and its modules or on impact assessments and other studies to support the further development of legislation and policy establishing a common EU area of criminal justice. 19

2.1.2Relevance of the programme to target group needs

A total of 70 out of 97 of respondents to the online survey indicated that they conducted needs assessments to support the design and development of their projects. However, further analysis of the project documentation of sixteen application forms showed that only six to have been developed on the basis of a needs assessment. Admittedly it was not a specific requirement to conduct and provide evidence backing up a needs assessment. However, if grant applicants had done so, this would have given a much clearer indication of the relevance of their project, enhanced the quality of their study and provided a baseline for assessing project success at a later date. Projects that are not based on needs assessment may still be relevant to the target groups’ needs, but not producing a needs assessment creates a risk that more relevant methods or means of supporting the target groups could have been developed. 20

Of the 97 respondents to the online survey, 69 (81.2 %) reported that they had received positive feedback from their target group(s) on the relevance of the project. One of the reasons for the reportedly high levels (81 %) of positive feedback from target groups may be because the (kind of) actor implementing the project was the same as the actors benefiting from it. More specifically, it can be presumed that members of the judiciary, legal professionals, etc. applied to provide training to their own colleagues alongside those from other Member States or to host meetings with fellow colleagues/experts from other Member States. Similarly, national authorities applied for grants to help them set up criminal records systems compatible with ECRIS and/or other e-justice facilities to harmonise their justice systems with those of other Member States. 21

2.2COHERENCE AND COMPLEMENTARITY

2.2.1Complementarity with other EU programmes

Article 12 of Council Decision No 2007/126/JHA establishing JPEN outlines the scope for complementarity and synergy creation with other EU financial instruments, namely the Civil Justice Programme (JCIV); the Prevention and Fight against Crime (ISEC); Prevention, Preparedness and Consequence Management of Terrorism and other Security Related Risks (CIPS); Solidarity and Management of Migration Flows (known as the ‘SOLID funds’); and the Community Statistical Programme.

Other EU instruments with the potential for complementarity or overlap with JPEN included the Daphne III Programme; the Fundamental Rights and Citizenship Programme (FRC); the Lifelong Learning Programme (LLP); the Seventh Research and Development Framework Programme (FP7); and Horizon 2020, the new EU Research and Innovation programme.

These programmes were similar to JPEN in objectives and themes. However, the fact that the JPEN objectives were specific made it easier to avoid overlap with other DG Justice programmes, which were different to JPEN in terms of:

• area of law covered — JPEN covers criminal law, JCIV covers civil law and FRC covers fundamental rights;

• focus — all focused on victims, but the Drug Prevention and Information Programme (DPIP) and Daphne III addressed specific groups of victims and considered prevention of harm and post-harm assistance. DPIP and Daphne III also focused more on treatment and protection than JPEN, which focused more on the victims’ journey through the criminal justice system;

• target groups or results — as in the case of ISEC, Daphne III and DPIP.

Also, the objectives of DG CONNECT's programmes and of FP7 have objectives that are coherent but not necessarily overlapping with JPEN. 22

The PROGRESS Programme and the European Social Fund (ESF) also funded the training of the judiciary, but each complemented JPEN: PROGRESS funded gender equality and anti-discrimination training for the judiciary (which none of the other justice programmes do); and the ESF mostly funded national training.

By contrast, the added value of the JPEN-funded training was that it was always transnational (as participants benefited from learning about other Member States and from increased transnational cooperation).

Table 2 1 below illustrates the scope for coherence and complementarity of JPEN’s objectives and thematic areas with the selected EU funding programmes.

Table 21 Scope for complementarity and overlap of objectives: JPEN and EU programmes

Judicial cooperationPromotion of compatibility of legislationExchange of information and best practices (in criminal justice)E-JusticeTraining of the judiciaryProtection of the rights of victims and the accusedCriminal justice
JPENJPENJPENJPENJPENJPENJPEN
JCIVJCIVJCIVJCIV
FRCFRC
ISECISECISEC
Daphne IIIDaphne III
DG Connect
FP7 / Horizon 2020


When preparing annual work programmes, complementarity and coherence between the DG Justice programmes were increased through Commission inter-service consultations and inter-service groups, where complementarity and possible overlaps in programming was discussed with other DGs. The Commission presented the annual work programme and its priorities to the Programme Committee for Member States to identify any complementarity and coherence issues with other EU or national initiatives. 23

2.2.2Complementarity at project level

At the project selection, monitoring and reporting stages of the programme cycle, the Commission applied no mechanisms to enhance complementarity, except for sharing resources (i.e. the e-Justice portal) with the Civil Justice Programme (JCIV).

At project level, grant beneficiaries sought to establish synergies with colleagues or other organisations implementing projects funded by JPEN or by other EU programmes; and/or with projects funded through national and international mechanisms.

Almost two thirds of respondents to the online survey indicated that some form of synergies were established with other projects under JPEN (29 %), other EU programmes (20 %), national programmes (20 %) or international programmes (5.5 %) 24 . However, not all respondents provided examples of the synergies they had created, so it was not always possible to corroborate this reported information.

Grant beneficiaries sought to increase the impact of their JPEN-funded projects by building on the findings of other projects; designing the project around good practices established under other programmes; and disseminating the results of the projects to networks established under other projects.

Among them, one beneficiary reported having disseminated the results of the project to end-user meetings of another project funded under DG Research’s FP7 programme. Another respondent explained that it had used some of the tools developed under a project funded by the LLP. Results of JPEN projects were also disseminated at conferences organised with funding from other EU programmes. 25

2.3EFFECTIVENESS

A programme is effective to the extent that it has succeeded in achieving its objectives. The success of JPEN can also be gauged by the extent to which individual JPEN projects succeeded in achieving their own objectives.

2.3.1Programme achievements

Overall, the implemented actions have addressed the objectives of the programme, in particular the specific objectives on training of the judiciary and judicial cooperation. It is difficult to assess the extent to which project outputs were achieved (in comparison to planned outputs). This is because a requirement for grant applicants to identify a measurable target for their outputs was only introduced towards the end of the programme. Based on the self-reporting of grant beneficiaries, it appears that most projects (around 70 %) were effective at achieving their outputs as proposed, while nearly two thirds managed to achieve all their outputs and results. 26

Overall JPEN action grants and operating grants produced outputs and positive results, in particular on 27 :

• training of the judiciary, which contributed to enhancing mutual trust, although its extent could not be measured, and benefited nearly 26 000 judges, prosecutors and other professionals in nearly all Member States;

• improved cooperation between the judiciary and justice ministries of Member States, with nearly a thousand partnerships created, most of which bilateral;

• greater compatibility of rules (or better understanding of the differences), through studies, mutual learning and the promotion of good practices aimed at harmonising rules. These also favoured the (better) implementation of EU legislation by increasing the knowledge and skills of those applying it at national level;

• addressing various obstacles to current judicial cooperation by developing appropriate services, research, mutual learning and networking. Types of obstacles addressed included: a lack of understanding and knowledge, communication channels and cooperation platforms, linguistic differences, etc.;

• increasing the capacity of organisations providing support to victims and the accused.

JPEN had a notable impact on improving judicial cooperation between the judiciary and justice ministries of Member States, not least because of the number of bilateral partnerships it created. At least 947 partnerships were formed, of which 826 were between Member States. Some of the outputs and results of transnational projects have included 28 :

• improved bilateral cooperation between Member State prosecutors on joint investigation teams in cross-border cases;

• increased expertise on the processing of EU nationals accused of crimes in another Member State;

• joint dialogue on common problems, also through the creation of cross-EU working groups such as the Legal Experts Advisory Panel.

JPEN clearly promoted judicial cooperation and compatibility between Member State rules and also contributed to reducing some legal obstacles.

The programme promoted the compatibility of rules between Member States by 29 :

• improving knowledge of differences between Member State legislation/practices and identifying possible solutions to reduce these differences;

• promoting good practices and greater harmonisation;

• increasing dialogue between those responsible for maintaining and amending rules on criminal justice, including the creation of working groups between Member States to discuss and identify solutions to obstacles in implementing EU legislation;

• promoting the (better) implementation of EU legislation by increasing Member States’ knowledge of it and capacity to implement it.

To reduce linguistic obstacles, JPEN identified access to interpretation services as a priority for funding, awarding grants for linguistic training of judges and prosecutors and the evaluation of linguistic services in Member States. Given the small scale of this linguistic training, it is unlikely that it had a major impact on reducing linguistic obstacles. However, it contributed to the training of judiciary and promoted the linguistic training of legal professionals 30 , which is a Commission priority 31 .

One of the key types of action funded (47 action grants plus 10 procurement contracts amounting to EUR 4.2 million) was support to Member States in amending their criminal records systems to make them compatible at EU level to enable the exchange of the records through ECRIS. Although project partners noted some implementation issues with the set-up of the ECRIS in Member States, ECRIS was successfully set up in 2012, meaning that the JPEN actions in this area were at least partly successful. 32

To the extent that JPEN actions supported the improvement of dialogue and exchanges between Member States and the fostering of judicial cooperation, the programme has improved contacts and exchange of information and best practices between legal, judicial and administrative authorities. At least 187 funded actions (out of 334) had mutual learning, the exchange of good practices and cooperation incorporated into their projects. These actions contributed to information exchange and the increasing of contacts between national members of the judiciary. National authorities from 21 Member States 33 led one or more JPEN grant, including 15 ministries of justice. Many of these grants were used to amend national criminal records systems to make them compatible with the European system (ECRIS). 34

In addition to actions funded through grants, JPEN funding was used to procure contracts with consultants who developed and maintained the e-Justice portal. 35

A total of 93 JPEN grants worth more than EUR 44 million (26 % of the total planned budget for the 2007-2013 programme) went specifically on training the judiciary and legal practitioners. This training benefited 25 863 judges, prosecutors and other professionals during the evaluated period, an average of 3 695 beneficiaries per year. Figure 2 1 shows the number of legal practitioners participating in these training events each year. This includes training funded through action grants, framework partnership agreements and operating grants to the EJTN.

Figure 21 Number of participants benefiting from JPEN-funded EU legal training 2007-2013


The Commission has set targets to train half of all EU legal practitioners (an estimated 700 000) by 2020. To do this, at least 5 % of all legal professionals in each Member State must be trained each year until 2020, whether the training is organised at local, national or European level or by local, national or EU-level actors 36 . A 2014 Commission report on European judicial training demonstrated that the target for judges in 2013 was reached (see top graph of Figure 2 2 ), while the target for annual training for prosecutors was reached in 19 Member States.

Figure 22 Number of judges and prosecutors participating in 2013 in continuous training in their own Member State and (for prosecutors) abroad



Source: European Judicial Training 2014. Figures include training funded through other EU programmes or by EU agencies, such as the PROGRESS, Civil Justice and Fundamental Rights and Citizenship programmes.

A large proportion of JPEN grants went to projects directly or indirectly focusing on the rights of victims and the accused. One of the main ways that JPEN contributed to building mutual trust on the rights of victims and of the accused in the EU is by funding EU-level platforms working in these areas, particularly:

• Fair Trials International, which seeks, among other actions, to ‘engage and support an international movement [network] of fair trials defenders [of human rights professionals]’; and

• Victim Support Europe (VSE), which is a network of national victim support organisations promoting the establishment and development of victim rights and services throughout Europe. 37

2.3.2Project achievements

Most projects (around 70 %) were effective at achieving their outputs as proposed and at achieving the project objectives. However, nearly a third were not as effective because they did not achieve all their objectives (see Table 2 2 ). Only in a handful of cases (around 5 % of all finalised projects) did this significantly affect the achievement of the project’s operational objectives. Although some of the obstacles encountered were due to unforeseen external factors, some could have been identified and mitigated as part of a risk assessment strategy.

Table 22 Drivers to the implementation issues experienced by projects

Driver to the problemNo. projects affected
Staff departure / insufficient staff5
Lack of time21
Unforeseen (external) obstacle17
Financial issues / lack of funding12
Technical reasons (e.g. problems with digital systems, with the functioning of e-justice software)5
Circumstances which should have been identified as part of the logic model (e.g. lack of data, lack of uptake from beneficiaries, timing issues, coherence with policy developments, etc.)15
Delays by the European Commission2
Changes to Logic model7

Source: all project documentation mapped by ICF (n = 334)


More than two thirds of the finalised projects were implemented exactly as planned, while one third made at least some changes to the way the projects/work programmes were implemented as compared with the original design. 38

2.4SUSTAINABILITY AND TRANSFERABILITY

The evaluators analysed whether the results, outcomes and impacts achieved by the projects were sustainable beyond the project funding period. Three levels of sustainability were identified in the evaluation:

1. short-term sustainability, achieved mainly through dissemination of projects’ results;

2. medium-term sustainability, implying continuation of project results and/or partnerships;

3. longer term sustainability, achieved mainly by transferring projects’ results to other contexts, organisations and Member States with little or no additional funding.

2.4.1Continuation of project activities and outputs and dissemination

Out of the 121 finalised projects demonstrating some evidence of sustainability in their final reports, 39 (32 %) stated in their final reports that they would continue the project in full because they had been able to secure further funding. These 39 projects dealt with the development of tools such as ECRIS, victim support services and restorative justice programmes — that is they dealt with systems or services which had been developed and which would require further funding for their continued implementation. However, this was not the case for all JPEN projects. Indeed, where the action grant or operating grant focused on research, the project would not necessarily require funding for its continuation. Similarly, projects that focused on training or mutual learning would only require further funding if the training were to be expanded to new participants. 39

A further 28 projects stated in their final reports that the project would continue in part, e.g. when aspects of the training continue, or when the findings resulting from the project are used to develop or contribute to new projects. In these cases, it can be assumed that funding has been identified for this part continuation. The results of the online survey suggested that most grant beneficiaries (71 %, or 61 out of 86 respondents) would require further funding to continue the project/activities. In follow-up interviews, 25 out of 33 interviewees stated that further funding was needed for follow-up aspects of the projects such as training and translations; testing of the products developed; and implementation of methodologies.

However, 20 out of 33 interviewees also stated that items such as dissemination activities and awareness raising could be continued without additional funding. 40

Overall, the dissemination of the projects’ results by the Commission was limited, since it mostly relied on its beneficiaries to disseminate the outputs directly to their target groups, and its own human resources were reserved mostly to the financial management. There was some publication of training materials, and the monitoring of project expenditure and budgets was prioritised over results monitoring. Only some outputs from training projects were published on the e-Justice portal. The training materials were only uploaded onto the e-Justice portal in cases where the products were comprehensive, relevant and fit a gap in coverage (mostly European Institute of Public Administration (EIPA) seminars, links to the EJTN and the ‘Building Mutual Trust’ 41 project websites). 42

Dissemination was a requirement of the calls for proposals and was encouraged by the Commission. The calls for action grants encouraged grant applicants to demonstrate a plan for dissemination. Operating grant applicants were not required to outline a strategy for dissemination. Out of 95 online survey respondents, 92 % (87 respondents) stated that they had a clear plan for disseminating the results of their project/activities. An analysis of 12 action grant application forms covering all priority areas and all years of the programme shows that dissemination strategies varied in quality. 43

The quantitative analysis of the documentation of 334 JPEN grants shows that at least 197 (59 %) of all grant beneficiaries implemented awareness-raising or dissemination activities as part of their projects/work programmes. The analysis of project outputs (see Figure 2 3 ) reported in grant beneficiaries’ final reports shows that the main tools for dissemination were events, newsletters or brochures and other outputs. Some grant beneficiaries also set up websites to disseminate project information, but at the time of the evaluation it was found that not all project websites were maintained once project funding had ended.


Figure 23 Outputs of dissemination and awareness-raising activities (n=221)


Source: analysis of 221 grants for which final reports were available.

However, the fact that such activities were effective at attracting the attention of policy-makers and increasing contacts between authorities in different Member States suggests that dissemination was at least somewhat effective. A total of 44 respondents to the online survey reported that they had already successfully implemented outputs in Member States other than their own. 29 also reported that the results of the project/activities had fed into policy activities in one or more other countries. 44

2.4.2Continuation of partnerships after the project’s completion

Based on the information in final reports, 17 partnerships functioning to implement JPEN grants would continue either in full or part after the project had ended. However, it is likely that the number of partnerships that continued was higher, because 71 out of 91 respondents to the online survey (78 %) stated that their partnership was already continuing or would likely continue after completion of the project. Another factor was that grant beneficiaries interviewed and participating in the online survey reported positively on the benefits of the partnerships formed under JPEN. Evidence from the follow-up interviews with grant beneficiaries suggested that some of these partnerships had already been formed before the start of the project. This previous partnership experience helped in facilitating team building, team communication and problem solving and made it easier to form initial contacts within the new project. 45 Data was not collected on the reasons of not continuing the partnerships.

2.4.3Potential sustainability and transferability of outputs

Disseminating outputs and results to a wide audience can increase their sustainability. Many of the outputs were already designed to be used at EU level (e.g. the EJTN’s judicial training was designed for participants from multiple Member States and the e-Justice portal for use by audiences in all Member States). Some of those that were not designed for cross-EU application were otherwise designed to be transferable to other Member States.

It was difficult for grant beneficiaries to demonstrate sustainability of their project in final reporting. This was because there were only two or three months between project closure and the reporting deadline, which is not really sufficient time to show sustainability.

Many of the project partners interviewed for the evaluation indicated that they had designed their projects around clear products, or outputs, many of which would or could be transferred to other Member States. A total of 70 out of 89 respondents to the online survey also expressed the view that all or some outputs could be used without any changes in more than one country, while an additional 22 reported that outputs could be transferred to more than one country with some minor changes. More than two thirds (61 respondents) also reported that the results of the project/activities could be fed into the policy activities in one or more other countries. 46

Factors that could be a barrier to the full transferability of the result were also identified by interviewees. These included adaptation to countries’ specific legislation; translation requirements; the maturity of other Member States’ judicial systems; the level of priority of a specific topic in a particular Member State’s agenda; and the need for a specific project output (i.e. training tool or method).

2.5EFFICIENCY AND SCOPE FOR SIMPLIFICATION

This section assesses the efficiency of the programme in terms of:

• the sufficiency and efficiency of financial resources available;

• the appropriateness of the money spent in comparison to the positive impacts achieved;

• the extent to which the allocation of funds among the different funding tools was appropriate and sufficient to implement the project’s objectives.

This section also reviews the Commission’s management efficiency and considers whether there is scope for simplifying the programme management.

2.5.1Sufficiency and efficiency of available financial resources

The analysis showed that the budget made available to achieve the objectives was sufficient, considering that the programme objectives were well expressed and not overly ambitious. Grant beneficiaries also reported that on the whole the grants were sufficient to enable them to realise their objectives and to make a difference in their area.

A comparison of allocated funds with (finally) committed funds suggests that project underspend was common towards the beginning of the programme: commitments per call were systematically lower than initial allocations, although some improvements occurred with later calls (see Figure 2 4 ). The reason for this initial mismatch was that JPEN was a new programme, which addressed a stakeholder group relatively inexperienced with funding applications. The funding tools used followed a clear logic and overall were used efficiently.


Figure 24 JPEN total allocated, committed and up-to-date paid funds and total over/under commitment and underspending by call for proposals (action grants above, operating grants below)






Note: The operating grant funding numbers also include the framework partnership agreements and operating grant monopolies. Figures on allocations and projects funded in the years 2011-2012 were merged. Underspending is calculated as the difference between committed and paid funding for all finalised projects. * Not all projects have been finalised (the spending and underspending figures only relate to finalised projects). CR-criminal records, EJ-e-justice, EJT-European Judicial Training.


2.5.2Money spent in comparison with the impacts achieved

JPEN provided EUR 131 million to fund 334 mostly transnational projects representing more than 1 200 lead and partner organisations 47 . In the end, with an additional EUR 21 million, JPEN financed 174 procured projects. This also contributed to the programme’s general and specific objectives, by for example supporting the preparation of legislation; bringing together experts to discuss certain obstacles; and communicating key messages. 48

The expected impacts of JPEN were to contribute to the emergence of a European area of justice in criminal matters, based on mutual recognition and mutual confidence. The expected outcomes related to reinforcement of judicial cooperation; support for European networks working to promote EU criminal justice; the further development of cross-border training for professionals; and a strengthening of mutual trust so as to protect the rights of victims and the accused. 49

While the expected impacts were certainly ambitious and challenging, the JPEN resources have made some contribution, although difficult to measure, to the wider EU objective of building a European area of justice by adding a dedicated funding stream which complemented a range of other means (e.g. legislation, policy, EU agencies). The programme also helped the Commission to realise several EU objectives which are part of the European area of justice. 50 The analysis of the finalised projects for which information was provided shows that as much as 73 % of the finalised 51 action grants and operating grants show evidence of obtained outcomes and impacts, whereas only 16 % showed no evidence of these 52 .

Although the evidence collected as part of the evaluation is still insufficient to firmly conclude that the resources spent on the programme were reasonable in relation to the outcomes and impacts achieved 53 , the outcomes and impacts identified would certainly suggest that spending was reasonable when looking at the programme’s achievements, as reported already under the ‘effectiveness’ criterion. 54

2.5.3Appropriateness of the allocation of funds among the different funding tools

According to the annual work programmes, the total budget allocations for implementing JPEN 2007-2013 were EUR 196 million, averaging EUR 28 million per year. The largest share of funding was allocated to action grants (EUR 115 million, or 58 % of the total programme value), followed by operating grants (EUR 50 million, or 25 %). Public procurement had an initial allocation of EUR 32 million, or 16 % of the total. However, when looking at the amounts allocated through the calls for proposals, action grants were allocated a total of EUR 123 million, which is around EUR 8 million more than what was initially planned in the annual work programmes (see Table 1 1 and Table 1 2 ). 55

The relatively low numbers of applications per call and their high success rates in particular for the first action grant calls (67 % for 2007 action grants, 87 % for 2007 action grants on criminal records-CR, 48 % for 2008 action grants and 77 % for 2008 action grants CR) again hint at the relatively low initial visibility of the programme and possible difficulties in attracting stakeholders to submit applications (see Figure 2 5 ).


Figure 25 Selected grants as a proportion of total applications per call


When looking at the absorption rates of action grants and operating grants (i.e. payments as a share of commitments), it appears that overall the granted budgets were slightly higher than the grant beneficiary could successfully absorb and deliver. The average absorption rate was 76 % for action grants and 80 % for operating grants. Operating grants also showed some improvements over time and hence a reduced underspend. On procurement, the data showed that out of the EUR 32 million initially allocated, EUR 21 million was committed. This would suggest that the funding made available was overall sufficient. 56

In addition, considering the particular stakeholder groups addressed, there is a risk of reaching their ‘full absorption capacity’ for this type funding. There are two reasons for this:

• the relatively ‘limited’ number of such stakeholders (in contrast to programmes such as Daphne III, for which there is much higher demand due to the much wider variety and higher numbers of stakeholders addressed);

• for many, the implementation of the grant is something done in addition to their ‘day-to-day’ activities.

2.5.4Commission management and scope for simplification

Based on the online survey results, requirements for applicants to access JPEN funding were deemed appropriate overall, as presented in Figure 2 6 below.

Figure 26 Responses on the online survey related to the perception of the Commission’s support during the application process



Source: Survey for the ex post evaluation of the five DG Justice programmes 2007-2013. Question 36: With regard to the Commission’s management of the five programmes, including the Commission’s monitoring and evaluation of your project/activities, please comment on the following statement: [statements in the graph]

The information provided in the calls for proposals was straightforward throughout the funding period, explaining the different elements relating to the application. However, the document became lengthier over time, partly reflecting the increased level of detail required in the application form, the number of priorities presented and the introduction in 2008 of the IT PRIAMOS system. The information included in the calls was clear and easy to understand according to most respondents to the online survey (i.e. 87 % of respondents answered ‘strongly agree’ and ‘agree’ to both statements). For the 2013 call, the single Guide for Applicants was in place, covering the five DG Justice programmes, ISEC and PROGRESS. The Single Guide contributed to simplification and efficiency for applicants submitting proposals for different projects. 57

Throughout the programme, the application form asked applicants to describe the project in terms of general project information; implementation; financial management; results; evaluation; and dissemination.

In addition, the applicants were required to complete budget estimation forms, staff-cost analysis and a partnership declaration. From 2010, applicants were also requested to provide indicators to assess results; provide evidence of previous programme experience; and add more detail on the partners and work streams.

This increased the potential for quality in the projects, motivating the applicants to develop a more rigorous plan and cost estimation of each of their activities.

The vast majority of respondents seemed to have been quite comfortable with the application process (i.e. with submitting the application and concerning the information included in the calls). However, when respondents were asked if they requested help from persons with specific expertise and knowledge on the procedures, only 45 % strongly to partially disagreed. 35 % of respondents knew of organisations/projects/practitioners that did not respond to the call for proposals due to the complex/difficult requirements, compared with only 21 % who said that they did not know of any such organisations/projects/practitioners. Overall, the reporting requirements (for both the financial aspects and the non-financial aspects of projects) were considered as appropriate by the vast majority of the respondents (81 out of 95, or 85 %). 58

The Commission’s monitoring arrangements were at least partially considered as good and helpful during the implementation of the project/activities by about 79 % of the respondents to the online survey (71 out of 90, or 79 %), while 21% were not fully satisfied (reasons are that the financial rules are quite complex, so the financial reporting becomes complicated for beneficiaries; the monitoring process is too bureaucratic; Commission staff sometimes gave differing answers throughout the life of the project). The requirement to submit a progress report was introduced in the 2011-2012 call. This made reporting more efficient. The final reporting was a fairly simple exercise, consisting of a narrative of the project’s results including problems encountered and methods used. 59

Overall, the Commission’s management became more efficient over time thanks to the introduction of the single unit dealing with programme management; the publication of a handbook for Commission officials; and indirectly, through the dissemination of a project management guide with the 2011 call.

Overall, grant beneficiaries’ experience of cooperation with the Commission was positive. The grant application requirements followed a similar process to other programmes managed centrally by the Commission and required more detailed information from the 2010 call onwards, which was beneficial for the quality of the applications and the projects. The changes to reporting also created a more balanced approach between financial justification on the one hand and evaluation/assessment of actual results and potential impacts of the projects on the other. 60

2.6EUROPEAN ADDED VALUE

EU added value refers to the extent to which the EU nature of the programme brings value to its stakeholders and the extent to which the EU has a comparative advantage over national and international actors working in the area. First, the EU nature of the programme and its geographical coverage is discussed. This is a starting point for identification of a EU added value. The EU added value is then analysed in terms of outputs that brought value to the EU and to beneficiaries. However, it has not been yet possible to identify and measure EU added value in terms of impacts.

JPEN had a strong transnational dimension. This is borne out by the objectives of the programme and by the kinds of actions deemed eligible, as stressed in the 2007 Council Decision. Article 4 of the Decision states that JPEN ‘shall support […] specific transnational projects of Union interest presented by at least two Member States or by at least one Member State and one other country which may either be an acceding or a candidate country.’ JPEN also involved a large number of EU platforms. Eligible actions also included national projects.

2.6.1Geographical coverage and involvement of Member States

All Member States participated in JPEN. Overall, there was quite good geographical coverage of activities across the EU. Some Member States received a larger number of grants and participated in a larger number of partnerships funded than others.


Figure 2 7 below presents the geographical location of organisations that participated in JPEN projects per Member State. Lead organisations were clustered within four Member States: Italy, Belgium, United Kingdom and Germany. Together these Member States led 48 % (161) of all projects. Many EU networks/platforms are registered in Belgium. No projects were led by organisations based in Croatia. Looking at the partner organisations, the Member State participation is more evenly spread: out of all Member States, 15 participated, involving more than 25 partner organisations.


Figure 27 Total number of organisations per Member State participating in JPEN, including lead (left) and partner (right) organisations



Furthermore, while lead organisations on average developed a higher number of partnerships with organisations from their own Member State than with organisations from other EU Member States 61 , up to 826 transnational partnerships were developed as a result of the programme 62 . Indeed, the average number of transnational partnerships per Member State was just under 32, with organisations based in Belgium, Germany, Italy, Spain and the United Kingdom each partnering with more than 100 organisations from other EU Member States.

In terms of partnership structure, Germany, Spain, United Kingdom, Belgium and Italy were the Member States whose organisations partnered with organisations in the highest number of different Member States, each having organisations partnering with peer organisations in 21-26 different EU countries. 63  

Figure 2 8 below highlights the partnership structure for projects led by Germany, Italy and the United Kingdom, since these were the three Member States with the highest number of lead organisations. German lead organisations partnered with 26 different EU Member States (other than Germany), while organisations from the United Kingdom partnered with 23 other Member States and organisations from Italy partnered with 21.


Figure 28 Partnership structure for the top three Member States in terms of lead organisations



2.6.2Added value for the EU and for grant beneficiaries

The EU added value of the programme for grant beneficiaries lay in the fact that provided them with access to funding to support them in implementing their obligations under EU law e.g. through training, tools and research to identify how obstacles to the implementation of EU law could be reduced, but also to get the necessary knowledge of good practices which could help their beneficiaries on the ground. The programme also facilitated judicial cooperation, which benefits those national authorities and members of the judiciary who may have to work on transnational cases or cases concerning EU criminal justice legislation. The programme also supported mutual learning between Member States. 64

Figure 2 9 shows the results of the online survey of grant beneficiaries’ opinions of the advantages of transnational partnerships in JPEN.


Figure 29 Aspects that the transnational partnership brought to the organisations implementing JPEN projects


Source: Survey for the ex post evaluation of the five DG Justice programmes 2007-2013. N=59. Question 29c: What did the transnational partnership bring to your organisation? (please select all that apply)


The transnational dimension of JPEN was also dependent to some extent on the good working relations established within the partnerships. The consultation showed that the partners were overall satisfied with the partnership formed in the context of JPEN projects. In particular, respondents were satisfied with the task allocation between partners, the number of organisations involved in the implementation of projects, the exchange of experience and lessons as well as with the overall communication. More than half of the survey respondents (62 %) actually indicated that it would have been useful to include more organisations from other Member States — indeed, this may have multiplied the beneficial impacts.


Out of 282 action grants, 40 were national. Of these, 33 focused on setting up criminal records systems that would be compatible with ECRIS. Three projects aimed to provide innovative ways of dealing with EU issues (i.e. a permanent support structure for judges working with the European arrest warrants, new methods for interpreting EU convictions and offences and a suicide prevention system for prison facilities) and four were studies mapping the application of EU legislation or training. The outputs were disseminated across the EU and were of clear EU value. 65


66 Finally, it can be assumed that a significant part of activities developed under JPEN would have not been developed had the programme not existed, since the programme had created a demand for new activities. The organisations involved were struggling to find national funding opportunities to maintain their activities, especially after the recent financial and economic crisis or develop other projects in the justice area. Most projects/activities did not receive further funding from other instruments. Moreover, 77 % of the online survey participants stated that the project/activities would not have been implemented without EU funding; only six people (6 %) indicated that it would have been implemented regardless of EU funding received.

3CONCLUSIONS

Relevance of the programme 67

• The specific objectives and priorities of the programme were largely specific, attainable and realistic, but they were not always measurable or time-bound.

• Because the priority areas of the calls for proposals were accompanied by a description of the expected content of the projects, this made them more specific and helped grant applicants to ensure that their projects were relevant to EU objectives. Furthermore, because the priority areas followed the programme objectives and EU policy objectives so closely, it was easy for the selected actions to target these objectives as well.

• Services procured using JPEN funding were also very relevant to the programme and wider EU objectives as they focused mainly on developing e-Justice tools (especially the e-Justice portal and its modules) or on research to support the development of legislation and policy.

• An analysis of grant application forms and information collected through consultation with grant beneficiaries shows that many grant beneficiaries either did not design their projects on the basis of a needs assessment or did not provide sufficient evidence to back up their assessment of needs in the grant application form. This creates a risk that more relevant or useful approaches to the project objectives might have been available.

• In spite of this, reporting by grant beneficiaries suggests that end beneficiaries responded positively to the projects, indicating that they considered the actions relevant. It is, however, not possible to corroborate this without gathering the independent views of end beneficiaries.

Coherence and complementarity

• Complementarity of JPEN with other EU programmes and interventions was almost fully achieved, although a few projects did risk overlap with the activities of other EU interventions.

• Complementarity was achieved through mechanisms that the Commission put in place at the programme design stage and at the stage of designing calls for proposals.

• At the project selection, monitoring and reporting stages of the programme cycle, the Commission applied no mechanisms to enhance complementarity, except for sharing resources (i.e. the e-Justice portal) with the Civil Justice Programme (JCIV).

Effectiveness

• Overall, the implemented actions addressed the programme objectives, in particular the specific objectives on the training of the judiciary and judicial cooperation.

• Projects that involved policy-makers in the project via consultation at design stage, through briefings, meetings or involving them on the project steering board contributed to achieving the programme’s intended outcome of having policy-makers use project outputs to shape new policy or legislation or adjust existing ones.

• It is difficult to assess the extent to which project outputs were achieved (in comparison to planned outputs). This was because the requirement for grant applicants to identify a measurable target for their outputs was only introduced by the Commission towards the end of the programme and final reports are not yet available for these projects.

• However, based on the self-reporting of grant beneficiaries (in final reports and through consultations conducted for this evaluation), it appears that most projects (around 70 %) were effective at achieving their outputs as proposed, but nearly a third were not as effective because they did not achieve all their objectives.


Sustainability

• Overall, the Commission’s efforts to disseminate the results of projects were limited except in relation to training materials, some of which were published on the e-Justice website. The Commission mostly relied on its beneficiaries to disseminate the outputs directly to their target groups, and its own human resources were reserved mostly to the financial management.

• It is difficult to fully assess the effectiveness of grant beneficiaries’ dissemination activities without consulting the target groups: information on the number of users and target groups reached in final reporting is not comprehensive, since the reporting of such information was not an obligation under the programme. Nonetheless, the fact that activities were effective at attracting the attention of policy-makers and increasing contacts between authorities in different Member States suggests that dissemination was at least somewhat effective.

• Of the 219 mapped action grants and operating grants for which final reports were available, 121 (55 %) demonstrated some evidence of sustainability, either because further project funding had been secured, because some activities would continue, because the partnership would continue (in part or in full) or because the outputs and results of the project would continue to be used.

• The proportion of projects (32 %) that identified further funding to continue the project (according to final reports) is seen as particularly positive. However, this may not be surprising given that the outputs produced as a result of JPEN projects were often targeted at national authorities and public services, which are well placed to identify follow-on funding for useful outputs.


Efficiency

• The funding made available to JPEN was sufficient and could possibly have been less for both action grants and operating grants, considering that it was a ‘new’ programme, focusing on a ‘new’ and relatively inexperienced stakeholder group. Commitments have systematically been lower than initial allocations, albeit showing some improvements.

• Based on the information available on funding, it seems that there was sufficient money available for grants to realise their objectives.

• As regards scope for simplification, the Commission’s management became more efficient over time and grant beneficiaries’ experience of cooperation with the Commission was positive.

• The level of detail required in the application form increased from the 2010 call onwards, which benefited both the Commission (in terms of quality and usefulness of the reports) and the applicants (allowing them to plan and estimate their activities more accurately). This increased detail included the introduction of work streams. Reporting requirements (for both the financial and non-financial aspects of projects) were considered as appropriate by the vast majority of the respondents (85 %). The Commission’s monitoring arrangements were at least partially considered as good and helpful during the implementation of the project/activities by about 79 % of the respondents to the online survey.


EU added value

• The EU nature of JPEN was present through a strong transnational dimension involving all EU Member States.

• The transnational partnerships also contributed to the achievement of the programme’s objectives, e.g. improving cross-border cooperation and contributing to the elaboration and dissemination of good practices, and, to a lesser extent, to more knowledge on relevant EU legislation and policies.

• The EU added value of the programme for grant beneficiaries lay also in the fact that the programme provided them with access to funding to support them in implementing their obligations under EU law. It was appropriate that the EU incentivised and facilitated the implementation of these obligations — and the achievement of EU objectives — through this fund.


Key recommendations

• Better define the priorities: the Commission should invest more time and human resources in the process of setting priorities in order to ensure that the priorities can be adequately achieved within an earmarked budget.

• Realistic assessments of project risks and better risk mitigation strategies: the Commission should better monitor risks throughout the project duration, for example by asking for brief progress reports that identify any potential risks as they arise during the implementation of the project.

• Increase focus on assessment of impacts at all levels and not merely on outputs, as regards monitoring and evaluation. This goes hand in hand with the need to collect, analyse and use objective and independent evidence in order to perform project and programme evaluations. Increase focus on needs assessment that each project aims to address.

• Explore ways of enhancing the uptake of project outputs, results and best practices by other organisations, including in other Member States, including more resources for translations, communication and dissemination. 

• Sharpen the programme's intervention logic; further to the scope of the programme and its general and specific objectives and priorities, types of action and types of intervention and implementing measures, the Commission shall seek to sharpen the intervention logic 68 , and make the relations between the rationale, objectives, inputs, outputs, beneficiaries, expected outcomes and impacts articulate, precise and concrete in any future continuation of the programme.


(1)

Article 16(3)(d), Council Decision No 2007/126/JHA of 12 February 2007 establishing for the period 2007 to 2013, as part of the General Programme ‘Fundamental Rights and Justice’, the Specific Programme ‘Criminal Justice’. http://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32007D0126&from=EN  

(2)

The ex post evaluation report by external evaluator is published here: Main report: http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jpen_programme_evaluation_final_report.pdf , Annexes: http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jpen_annex_1_quantitative_analysis.pdf , http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jpen_annex_2.pdf

Mid-term evaluation report by external evaluator is published here: http://ec.europa.eu/justice/funding/jpen/interim_evaluation_report_2011_en.pdf  

(3)

Article 7, Council Decision No 2007/126/JHA of 12 February 2007 establishing for the period 2007 to 2013, as part of the General Programme ‘Fundamental Rights and Justice’, the Specific Programme ‘Criminal Justice’, http://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32007D0126&from=EN  

(4)

Article 6, Council Decision No 2007/126/JHA of 12 February 2007 establishing for the period 2007 to 2013, as part of the General Programme ‘Fundamental Rights and Justice’, the Specific Programme ‘Criminal Justice’, http://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32007D0126&from=EN  

(5)

According to the Article 13, Council Decision No 2007/126/JHA of 12 February 2007 establishing for the period 2007 to 2013, as part of the General Programme ‘Fundamental Rights and Justice’, the Specific Programme ‘Criminal Justice’, The budgetary resources allocated to the actions provided for in the Programme shall be entered in the annual appropriations of the general budget of the European Union. The available annual appropriations shall be authorised by the budgetary authority within the limits of the financial framework.

(6)

The operating grant allocations also include money allocated to monopolies and framework partnership agreements.

(7)

 Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Criminal Justice Programme (JPEN), ICF, 28 July 2015, p. 2 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jpen_programme_evaluation_final_report.pdf

(8)

Ibid.., p. 4

(9)

Ibid., p. 4

(10)

Ibid., p. 5

(11)

Ibid., p. 7

(12)

For instance, a ‘Legal Experts Advisory Panel’ project discussed obstacles to judicial cooperation/mutual confidence in the EU and recommended ways to improve this.

(13)

 Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Criminal Justice Programme (JPEN), ICF, 28 July 2015, p. 9 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jpen_programme_evaluation_final_report.pdf

(14)

Ibid., p. 11

(15)

Ibid., pp. 10-11

(16)

Ibid., p. 11

(17)

Ibid.

(18)

Ibid.

(19)

Ibid., p. 5

(20)

Ibid., pp.13-14

(21)

Ibid., p. 14

(22)

Ibid., p. 16

(23)

Ibid., p. 20

(24)

26 respondents of the survey (or 27 % of all respondents) stated that no synergies were established and 11 gave the response ‘I do not know’. There is a slight methodological issue with this data: not all respondents had the option of replying ‘no’ or ‘I don’t know’, because these two categories where only added after the survey was already running. A total of 17 of the respondents considered in this report did not have this option and therefore of the 33 not providing a response to this question, 17 may have replied ‘no’ or ‘I do not know’ had they been given the option.

(25)

 Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Criminal Justice Programme (JPEN), ICF, 28 July 2015, p. 21 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jpen_programme_evaluation_final_report.pdf

(26)

Ibid., p.22

(27)

Ibid., p. 45

(28)

Ibid., p. 26

(29)

Ibid., p. 27

(30)

Ibid., pp. 27-28

(31)

See the 2011 Communication on Building trust in EU-wide justice: a new dimension to European judicial training http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2011:0551:FIN:EN:PDF .

(32)

 Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Criminal Justice Programme (JPEN), ICF, 28 July 2015, p. 28 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jpen_programme_evaluation_final_report.pdf

(33)

All except for Belgium, Cyprus, Denmark, Hungary, Malta, Sweden and Slovakia.

(34)

 Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Criminal Justice Programme (JPEN), ICF, 28 July 2015, p. 27 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jpen_programme_evaluation_final_report.pdf

(35)

Ibid., p. 28

(36)

COM(2011) 551 final ‘Building trust in EU-wide justice: a new dimension to European Judicial Training: http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2011:0551:FIN:EN:PDF.

(37)

 Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Criminal Justice Programme (JPEN), ICF, 28 July 2015, p. 28 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jpen_programme_evaluation_final_report.pdf

(38)

Ibid., p. 31

(39)

Ibid., p. 36

(40)

Ibid.

(41)

The project supports linguistic training for legal professionals.

(42)

 Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Criminal Justice Programme (JPEN), ICF, 28 July 2015, p. 33-34 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jpen_programme_evaluation_final_report.pdf

(43)

Ibid., p. 34

(44)

Ibid., p. 35

(45)

Ibid., p. 36

(46)

Ibid., p. 37

(47)

This figure does not account for double counting of organisations who received more than one grant.

(48)

 Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Criminal Justice Programme (JPEN), ICF, 28 July 2015, p. 45 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jpen_programme_evaluation_final_report.pdf

(49)

Ibid., p. 45

(50)

Ibid.

(51)

In total, 210 action grants and 50 operating grants are considered to be nearly or fully finalised.

(52)

For the remaining projects no clear evidence was available or there was no information.

(53)

Also considering that around one third of the actions funded were still to be completed and that it takes time for certain outcomes and impacts to be realised.

(54)

 Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Criminal Justice Programme (JPEN), ICF, 28 July 2015, p. 45 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jpen_programme_evaluation_final_report.pdf

(55)

Ibid., p. 40

(56)

 Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Criminal Justice Programme (JPEN), ICF, 28 July 2015, p. 42 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jpen_programme_evaluation_final_report.pdf

(57)

Ibid., p. 49

(58)

Ibid.

(59)

Ibid., p. 39

(60)

Ibid.

(61)

This is especially noticeable for Italy. For instance, Italian lead organisations partnered with 28 Italian partners, 21 Spanish partners and 13 French partners. UK lead organisations partnered with 14 Dutch partners, 13 UK partners and 12 Spanish partners. German lead organisations partnered with 12 German partners, 11 Spanish partners and 9 Austrian partners.

(62)

Based on a calculation of the number of non-national partners each lead organisation partnered with. This number is likely to be overly high as it does not account for partnerships involving the same organisations that were used for more than one project.

(63)

 Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Criminal Justice Programme (JPEN), ICF, 28 July 2015, p. 54 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jpen_programme_evaluation_final_report.pdf

(64)

Ibid., p. 57

(65)

Ibid.

(66)

Ibid., pp. 57-58

(67)

Ibid., pp. 59-61

(68)

See for instance Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Criminal Justice Programme (JPEN), ICF, 28 July 2015, pp. 1-2 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jpen_programme_evaluation_final_report.pdf .