Annexes to COM(2017)59 - Ex post evaluation report on the Civil Justice Programme (2007-2013) - Main contents
Please note
This page contains a limited version of this dossier in the EU Monitor.
| dossier | COM(2017)59 - Ex post evaluation report on the Civil Justice Programme (2007-2013). |
|---|---|
| document | COM(2017)59  |
| date | February 13, 2017 |
In addition to added value for grant recipients, the survey indicates that the projects added value by contributing to achieving the EU objectives. Most survey respondents reported that the implemented project/activities made a significant/major contribution to improving cross-border cooperation and the elaboration and dissemination of best practices. Among these, analytical activities mainly contributed to other objectives (40%), mutual learning mainly contributed to developing mutual trust among countries (28%) and awareness-raising mainly contributed to the elaboration and dissemination of best practices (25%).
However, most respondents reported that the implemented project/activities only made a minor or no contribution to creating practical tools and solutions that address cross-border or Union-wide challenges and to making a wider public aware of rights deriving from EU law.
The EU added value is particularly evident in the responses of grant beneficiaries concerning how important EU funding was to achieving the objectives. 87% of survey respondents stated that the projects/activities would not have been implemented without EU funding. It can therefore be assumed that most of the project/activities carried out under the JCIV programme would not have been implemented had the programme not existed. 61
3CONCLUSIONS
Relevance of the programme 62
• The programme objectives were largely specific, attainable and realistic. However, they were neither measurable, nor time-bound (although bound by programme and project duration).
• Overall the priorities and funded actions can be considered relevant to the objectives of the programme. However, the process put in place to select the programme’s priorities left little room for manoeuvre, once the priorities had been set (although in principle a project which did not reflect any priority could receive a grant under other award criteria). This meant that in some cases, if a priority changed during the implementation of a project, its results were less useful. Conversely, with regard to training, for example, needs could arise that did not fall under the set priorities. They could not then be funded under the programme because they were not taken into consideration in a specific call.
• The JCIV programme was well designed to support the practical application and implementation of EU policy and legislation, and to contribute to policy and legislative development.
• Overall, grant beneficiaries found that the calls and selected actions identified, and responded to, the needs of the target groups, although the extent to which they adequately identified these needs varied, with some undertaking elaborate needs assessments and others adopting a less rigid approach. However, the robustness of the method used for the needs assessments carried out by the projects could not be assessed by this evaluation. Needs assessments are not a mandatory requirement for the grant beneficiaries.
Coherence and complementarity
• Overall, the JCIV achieved a certain degree of complementarity with other EU programmes in terms of certain objectives and thematic areas, the nature of the programme and the target groups, especially with the JPEN and FRC. At the same time, however, there is a risk of overlap with these programmes.
• Synergies were established at EU and national levels and within the programme itself.
• The JCIV programme was consistent with policy developments at EU and national levels and was able to support policy developments and legislation in civil justice through the actions funded and through public procurement contracts.
Effectiveness
• Overall, the actions funded made a contribution to the programme objectives. However, it is difficult to assess the effectiveness of the programme because of the limited information on outcomes and impacts provided in the projects final reports and the interviews. The Commission's monitoring and evaluation system was rather output-oriented than results-oriented which explains some of the deficiencies of the present evaluation report.
• Most of the projects were able to achieve their planned results on time and reach the expected target group. No obstacles seriously influencing the implementation of the project’s results or objectives were identified.
• The JCIV has contributed to the implementation and development of EU policy and legislation, via grants and public procurement contracts. However, the lengthy project cycle could nullify the effectiveness of the finalised actions where policy priorities changed.
Sustainability
• Regarding short-term sustainability (i.e. through dissemination of project results), the Commission’s efforts to disseminate (and monitor) project results have been limited, overall. The Commission mostly relied on its beneficiaries to disseminate the outputs directly to their target groups, and its own human resources were dedicated mostly to the financial management. Dissemination to a wider public is identified as key element in giving the programme greater impact. Dissemination of JCIV results was limited in effectiveness by the lack of a specific channel for disseminating them. Grant beneficiaries’ dissemination appeared to be more efficient, as most had established clear plans for disseminating the results of their project/activities.
• Overall, the JCIV programme generated sustainable results in the medium (i.e. continuation of project results and/or partnerships) and long term (i.e. through the transfer of projects’ results to other contexts, organisations and Member States with little or no additional funding). Regarding the continuation of partnerships, the evidence collected is somewhat contradictory: while the JCIV final reports suggested that this was low, the online survey showed a different result, with most respondents indicating that the partnership had continued. This could be due to partnerships being ‘reconstituted’ only after a while, when the final report had already been submitted.
• Most projects were considered innovative, particularly in targeting groups or tackling issues that had not been researched or addressed before, or through the development of new methodologies, approaches or tools.
• More than half of survey respondents indicated that further funding was needed to guarantee financial sustainability of the project/activities.
Efficiency
• Overall, funding provided to action grants, operating grants and procurement contracts appears to have been sufficient to support the achievement of the JCIV’s general objectives.
• The financial resources have been used efficiently overall, judging by the comparison of inputs and outputs between projects’ budgets, and the total outputs and results produced. In the first years of implementation, absorption rates were relatively low, pointing to a lack of financial and administrative capacity of the potential beneficiaries, but this increased to an acceptable level after two years.
• The Commission’s management of the JCIV was perceived as efficient in terms of the requirements imposed on applicants and grant beneficiaries and of the support received. However, some organisations found the application and implementation process difficult. There is some scope for simplifying procedures for applicants and beneficiaries (e.g. improving technical/IT systems or simplifying accounting procedures and financial reporting). Reporting arrangements were considered appropriate overall, but sometimes cumbersome, particularly with regard to financial reporting. The Commission’s monitoring arrangements were considered useful overall.
EU added value
• The JCIV programme had a strong transnational dimension. The transnational partnerships resulted in specific benefits for the organisations involved in implementing JCIV activities, e.g. an increased knowledge base of participating organisations, networking consisting of (more) international partners and improved knowledge of policy and practice in other countries.
• The geographical coverage of action grants and operating grants was uneven, with a few Member States overly represented (Belgium, Germany, France and Italy) and others involved little or not at all.
Key recommendations
• Better define the priorities: the Commission should invest more time and human resources in the process of setting priorities in order to ensure that the priorities can be adequately achieved within an earmarked budget.
• Realistic assessments of project risks and better risk mitigation strategies: the Commission should better monitor risks throughout the project duration, for example by asking for brief progress reports that identify any potential risks as they arise during the implementation of the project.
• Increase focus on assessment of impacts at all levels and not merely on outputs, as regards monitoring and evaluation. This goes hand in hand with the need to collect, analyse and use objective and independent evidence in order to perform project and programme evaluations. Increase focus on needs assessment that each project aims to address.
• Explore ways of enhancing the uptake of project outputs, results and best practices by other organisations, including in other Member States, including more resources for translations, communication and dissemination.
• Sharpen the programme's intervention logic; further to the scope of the programme and its general and specific objectives and priorities, types of action and types of intervention and implementing measures, the Commission shall seek to sharpen the intervention logic 63 , and make the relations between the rationale, objectives, inputs, outputs, beneficiaries, expected outcomes and impacts articulate, precise and concrete in any future continuation of the programme.
(1)
Article 16(3)(d) of Decision No 1149/2007/EC of 25 September 2007 establishing for the period 2007-2013 the Specific Programme 'Civil Justice' as part of the General Programme 'Fundamental Rights and Justice'.
(2)
The ex-post evaluation report by external evaluator is published here: Main report: http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jciv_programme_evaluation_final_report.pdf , Annexes: http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jciv_annex_4_quantitative_analysis.pdf
Mid-term evaluation report by external evaluator is published here: http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2011:0351:FIN:EN:PDF
(3)
These two sources are rather subjective sources of evidence, and cannot be taken for independent evidence of performance.
(4)
Article 8 of Decision No 1149/2007/EC of 25 September 2007 establishing for the period 2007-2013 the Specific Programme 'Civil Justice' as part of the General Programme 'Fundamental Rights and Justice'.
(5)
Ibid.
(6)
Ibid.
(7)
Article 13 of Decision No 1149/2007/EC of 25 September 2007 establishing for the period 2007-2013 the Specific Programme 'Civil Justice' as part of the General Programme 'Fundamental Rights and Justice'.
(8)
Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Civil Justice (JCIV), ICF, 28 July 2015, p. 38 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jciv_programme_evaluation_final_report.pdf
(9)
Budget allocated to action grants for ‘specific trans-national projects of Community interest presented by an authority or any other body of a Member State, an international or non-governmental organisation, and involving in any case at least two Member States or at least one Member State and one other State which may be either an acceding country or a candidate country’ (Article 4(b) of the founding Decision).
(10)
Budget allocated, according to the annual work programme, to action grants for specific projects to help implement European competition rules. These are projects to promote judicial cooperation between, and training of, national judges in the context of the enforcement of the European competition rules, including Articles 101 and 102 of the TFEU and the State aid rules, with the aim of contributing to the development and implementation of European competition policy and Community cooperation measures in the field of competition to ensure the consistency of Community competition policy.
(11)
Budget allocated to operating grants for ‘the activities of non-governmental organisations or other entities pursuing an aim of general European interest in accordance with the general objectives of the Programme under the conditions set out in the annual work programmes’ (Article 4(c) of the founding Decision).
(12)
Budget allocated to ‘operating grants to co-finance expenditure associated with the permanent work programmes of the European Network of Councils for the Judiciary and the Network of the Presidents of the Supreme Judicial Courts of the European Union, insofar as it is incurred in pursuing an objective of general European interest by promoting exchanges of views and experience on matters concerning case-law and the organisation and functioning of the members of those networks in the performance of their judicial and/or advisory functions with regard to Community law’.
(13)
Budget allocated to the management of the membership of the community in the Hague conference on private international law.
(14)
Budget allocated, according to the annual work programme, for operation of the European Judicial Network in Civil and Commercial Matters, created by Council Decision No 2001/470/CE of 28 May 2001, establishing a European Judicial Network in Civil and Commercial Matters.
(15)
Budget allocated to ‘specific actions initiated by the Commission, such as studies and research, opinion polls and surveys, formulation of indicators and common methodologies, collection, development and dissemination of data and statistics, seminars, conferences and expert meetings, organisation of public campaigns and events, development and maintenance of websites, preparation and dissemination of information material, support for and management of networks of national experts, analytical, monitoring and evaluation activities’ (Article 4(a) of the founding Decision).
(16)
Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Civil Justice (JCIV), ICF, 28 July 2015, p. 3 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jciv_programme_evaluation_final_report.pdf
(17)
Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Civil Justice (JCIV) quantitative analysis, 28 July 2015, p. 10 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jciv_annex_4_quantitative_analysis.pdf
(18)
SMART objectives are defined as being specific, measurable, achievable, results-focused and time-bound.
(19)
Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Civil Justice (JCIV), ICF, 28 July 2015, p. 7 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jciv_programme_evaluation_final_report.pdf
(20)
European Commission communication Building trust in EU-wide justice — A new dimension to European judicial training, COM/2011/0551 final, 13.09.2011.
(21)
E.g. to the EJTN via the JPEN programme and to other organisations, such as the Academy of European Law (ERA) and the European Institute of Public Administration (EIPA), through operating grants from the Directorate-General for Education and Culture.
(22)
Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Civil Justice (JCIV), ICF, 28 July 2015, p. 7 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jciv_programme_evaluation_final_report.pdf
(23)
Regulation (EU) No 650/2012 on jurisdiction, applicable law, recognition and enforcement of decisions and acceptance and enforcement of authentic instruments in matters of succession and on the creation of a European Certificate of Succession.
(24)
Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Civil Justice (JCIV), ICF, 28 July 2015, p. 7 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jciv_programme_evaluation_final_report.pdf
(25)
Ibid., p. 12
(26)
Ibid., p. 13
(27)
Ibid., p. 12
(28)
Ibid., p. 14
(29)
Regulation (EU) n. 606/2013 on mutual recognition of protection measures in civil matters.
(30)
Directive 2011/99/EU on the European protection order.
(31)
European Commission website, Fundamental Rights and Citizenship Funding Programme, available at http://ec.europa.eu/justice/grants/programmes/fundamental-citizenship/index_en.htm .
(32)
Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Civil Justice (JCIV), ICF, 28 July 2015, pp. 14-15 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jciv_programme_evaluation_final_report.pdf
(33)
Ibid., p. 8
(34)
Ibid., p. 17
(35)
Ibid., p. 18
(36)
Ibid.
(37)
Ibid., p. 19
(38)
Ibid., p. 19
(39)
Ibid., p. 23
(40)
Ibid.
(41)
Ibid., p. 32
(42)
Ibid.
(43)
Ibid., p. 33
(44)
Ibid., p. 32
(45)
Ibid.
(46)
Ibid., p.27
(47)
Ibid., p. 31
(48)
Ibid., p. 27
(49)
Ibid., p. 34
(50)
Ibid.
(51)
Ibid.
(52)
Ibid.
(53)
Ibid., p. 37
(54)
Ibid., p. 39
(55)
Ibid.
(56)
Ibid.
(57)
Ibid.
(58)
Ibid.
(59)
87% of survey respondents to this question stated that the projects/activities would not have been implemented without EU funding.
(60)
Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Civil Justice (JCIV), ICF, 28 July 2015, p. 52 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jciv_programme_evaluation_final_report.pdf
(61)
Ibid., p. 53
(62)
Ibid., pp. 54-56
(63)
See for instance Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Civil Justice (JCIV), ICF, 28 July 2015, pp. 1-2 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/jciv_programme_evaluation_final_report.pdf .