Annexes to COM(2016)243 - Ex-post evaluation of the 2nd Health programme 2008-2013 under Decision No 1350/2007/EC establishing the programme of Community action in the field of health (2008-2013)

Please note

This page contains a limited version of this dossier in the EU Monitor.

agreements€13 805 987.005 %Grants for conferences€5 268 308.142 %Other 5€11 693 227.814 %Total€293 902 957.87100 %

2.3.    Programme outcomes

The Programme generated useful knowledge and evidence to serve as a basis for informed policymaking and further research. This took the form of best practice, tools and methodologies that help to secure benefits for both the public-health communities and citizens directly (e.g. with regard to improving diagnostic tests, supporting Member States in developing national actions plans on cancer, improving patient care, etc.). It supported awareness and sustained networking activities (e.g. by co-funding pan-European conferences and networks such as those in the field of public health and health promotion). Also, it supported the collection of comparable data across the Union, covering many Member States and providing information for policymaking purposes, e.g. European core health indicators (ECHIs) and the ORPHANET database on rare diseases, and produced training/educational materials (e.g. to train health professionals on migrant and ethnic minority health) and guidance. Finally, it supported public-health capacity-building at various levels (e.g. by fostering Member States’ preparedness in the event of health emergencies) through training and exchange of knowledge between healthcare institutions in the Member States.


3. Programme evaluation

External independent evaluations were conducted at mid-term (in 2010-2011) and ex-post (2014-2015). Both gave a positive assessment of the Programme with limitations. They confirmed the relevance of the actions funded, although underlining the weaknesses in the Programme design (large scope and non-explicit specific objectives). They concluded that the Programme demonstrated EU added value mainly regarding identification of best practices, benchmarking for better decision making and networking but to a lesser extent on innovation, EU health legislation and economics of scale, while the actions covering cross-border health threats and free movement of persons were under-represented. They concluded that every relevant need cannot lead to an impactful action; only actions that have an EU added value have the potential to impact on Member States’ health policies. The impact depends on the uptake and implementation of the results by the Member States. Therefore the dissemination to the relevant stakeholders, in particular to national health authorities, that set out and implement national health policies, is important. In relation to outreach to relevant stakeholders the actions of the Programme were successful to varying degrees. Another factor that is influencing the uptake by Member States is their buy-in on the actions which was increased through a stronger focus in the second half on Joint Actions.

3.1.    Overall assessment

The 2nd Health Programme reflects the development of EU health policy, with more emphasis — in addition to the classical publichealth approach (promotion and prevention of diseases, fight against communicable diseases and other cross-border health threats, safety of patients) — on other health issues emerging from the need to innovate and modernise health systems while keeping health expenditure under control, and to optimise actions in areas such as health technology assessment (HTA), e-health and the cross-border directive 6 for the health and rights of EU citizens. The Programme mobilised relevant stakeholders in all 28 Member States (with more of them being from EU 15 than EU 12 Member States) and three EEA EFTA countries (Norway, Iceland and Lichtenstein). It covered a series of common health concerns (i.e concerning major chronic diseases, the health of an ageing population, the development of capacity to respond to cross-border health threats), provided a wide range of useful tools (i.e the EU information database for rare diseases) 7 and best practices (i.e. for preventing the spread of HIV/AIDS 8 , addressing obesity 9 or applying cancer screening) for use at national and regional levels, and promoted cooperation and coordination between Member States’ health authorities to keep health issues such as health inequalities 10 , mental health 11 , ageing, etc. high on their agenda.

However, both evaluations pointed out that the lack of explicit objectives and progress indicators in the Programme's design was conducive to the proliferation of priorities and made it difficult to define the overall results in a meaningful way.

3.2.    Mid-term evaluation 12

The mid-term evaluation included a number of recommendations to improve the prioritisation of needs, fix more targeted and focused objectives, co-fund fewer actions, monitor on the basis of SMART indicators and better disseminate outcomes.

The pertinent recommendation for streamlining the annual priorities, providing better guidance to applicants and ensuring better dissemination of actions and their results, were implemented immediately in the 2011–2013 annual work programmes. This resulted in more focus (limiting the health topics covered) and better alignment with the Europe 2020 strategy. The CHAFEA improved the guide for applicants and the help-desk services for the calls and concentrated more on the dissemination of results.

The lessons learnt from the evaluation were applied in the impact assessment accompanying the Commission’s proposal for the 3rd Health Programme and guided the design of the new Programme. As a result, Regulation (EU) No 282/2014: 13

provides for explicit specific objectives and indicators (Article 3);

establishes a restricted number of clear thematic priorities (Annex I);

includes specific criteria for gauging EU added value (recital 6) concerning the prioritisation of needs (Annex II) and the awarding of funding;

formally recognises the role of national focal points (Article 15); and

underlines the need for better dissemination of the results of actions and overall corporate communication on the Union’s political priorities (Articles 13(4) and 9).

The mid-term evaluation of the 3rd Health Programme, will assess whether and to what extent the above changes contribute effectively to improved implementation and the achievement of Programme objectives.

3.3.     Ex post evaluation 14

The ex post evaluation reviewed key aspects of Programme implementation, including followup of the mid-term review recommendations. It focused on the management of the Programme, the dissemination of its results, the Programme's effectiveness, and synergies with other EU programmes. This meant that it was possible to concentrate on issues that could not yet have been sufficiently explored in the mid-term evaluation and avoid duplicating previous evaluation work. Given the fact that in health it usually takes a long time for results and impacts to manifest, and that in any case the impact of the Programme depends on the uptake of actions' outcomes by Member States, which is out of the Programme's control, evaluators approached the issue with a view to the next programming period and sought to elicit elements for a better understanding of how the Programme could impact on health policies in the Member States. The conclusions identified both success factors and factors that negatively influence the potential impact of actions funded under the 2nd Health Programme. Those lessons learnt will be used for improving the implementation of the 3rd Health Programme and will feed into the three following main areas of action below.

(a)Improved monitoring, reporting and dissemination efforts

The management of the Programme was substantially improved. Programme priorities were streamlined in the second half of the period around the Europe 2020 initiatives and its objectives for smart, sustainable and inclusive growth. Monitoring data were (and continue to be) collected and a more systematic approach for their collection, analysis and regular use will provide an improved evidence basis for setting of annual priorities, for reporting 15 on outputs and results, and for better targeting dissemination to relevant stakeholders.

The success of the Programme depends to a large extent on the willingness of Member States and other participating countries to take up the results and integrate them into national initiatives and policies (i.e. the guide for cancer screening now adopted by all Member States; the preparedness plans of Member States for responding to health threats and emergencies). It is therefore important that the actions and their results are well known to the relevant key stakeholders. This is key for the Programme’s visibility, accountability as regards the use of its resources and above all its usefulness.

The Commission has intensified dissemination activities in the past two years. The main dissemination tools have been CHAFEA’s project database, 16 brochures 17 presenting the cofunded actions by policy theme (e.g. health inequalities, rare diseases, etc.), the regular organisation of ‘cluster meetings’ 18 with journalists (e.g. on organ transplantation 19 and HIV/AIDS) 20 and a highlevel conference on the Programme in 2012. 21 Since 2014, the Programme has supported Member State authorities in the organisation of conferences and other events 22 promoting the dissemination to key audiences of Programme results on specific health topics. This intensified support for the dissemination efforts of individual actions has shown to be useful and to help extend the reach of relevant results. However, in light of the diversity and breadth of action, they cannot make up for the fact that some projects have not promoted their results to the specific audiences they are most relevant for. Therefore, in the 3rd Health Programme an effective dissemination strategy must be defined early on for all actions.

(b)Encouraged participation of all Member States and other participating countries

The Programme is an instrument to support policy development, coordination and implementation in the area of health. While competence in this area lies largely with Member States, the Programme helped them to develop initiatives at EU level for more effective and efficient solutions to common health concerns, e.g. the ‘European Guide for Quality National Cancer Control Programmes’, which provides an outline for policy-makers on the basic tenets of cancer control policy; better knowledge of the quality and effectiveness of the systems for organ donation and transplantation in participating countries; diagnostic tools and a laboratory network to boost Member States’ preparedness and cooperation in the event of health threats; or the rights of patients living and working in a Member State other than their own.

All Member States, the three EEA EFTA countries (Iceland, Liechtenstein and Norway) and Croatia participated in the Programme to varying degrees. Despite the fact that EU 15 Member States have participated more than the EU 12 Member States in the programme in general, and as coordinators of actions in particular, the Health Programme involved Member States with low GDP/GNI more than the FP7. The participation of as many countries as possible creates the necessary networking environment to identify common health concerns, raise awareness of key emerging health problems and transfer knowledge in order to improve health status and address inequalities. Health can be a smart incentive for investment 23 in the economy and plays a key role in European cohesion and inclusion.

Difficult economic circumstances are stated as among the reasons for some Member States' low participation. To address this the Commission has included the principle of ‘exceptional utility’ in the current 3rd Health Programme 24 as a financial incentive to involve all Member States. Information days to promote the Programme should continue to be organised in all Member States and participating countries, the aim being that at least half of these events will take place in Member States underrepresented in the previous Programme period. More information material should be available in all EU languages.

The Programme is also open on a cost basis to acceding, candidate and potential candidate countries, where it functions as a mechanism that can help countries adapt to the acquis communautaire in the health area through their cooperation with Member States in projects and joint actions.

(c)Implementation of the Programme in synergy with other European programmes and Commission priorities

The uptake of Programme outcomes and their integration into national policies remains the determining factor whether or not the Programme is a success. Given the limited budget and the tremendous health needs and challenges, developing synergies with other programmes to spread its outcomes further, promote their use and build on them has already been more extensively pursued with the Union research Programmes (FP7 and Horizon 2020) and to a lesser degree with the European Structural and Investments Funds (ESIFs).

Breaking down ‘silos’ and working more closely with other Union programmes is part of the Commission’s vision for delivering on its 10 major priorities 25 in a coordinated way. The 3rd Health Programme offers possibilities for such cooperation in supporting health in areas such as migration, dealing with emerging diseases relating to environmental factors such as climate change, innovation in health, healthsector reform and global health. Finally, the Programme involves assessments of health systems and generates country-specific and cross-country knowledge to inform policies at national and European level.


4. Conclusion

In the coming years, the Commission will make a particular effort to improve monitoring, reporting and dissemination, support the participation of less active Member States in the Programme and develop synergies with other EU programmes. The results and ideas emanating from the Programme could be communicated more actively through existing networks of contacts in health ministries and via coherent and comprehensive coordination with actors in Union programmes that cover, but do not deal exclusively with, health.


After 18 years of EU Health Programmes, at the end of the 3rd Health Programme in 2020, the Commission will assess the extent to which the first three Programmes have produced positive impacts on Member States health policies in an effective, efficient, coherent and relevant way ensuring EU added value. The monitoring arrangements for the 3rd Health Programme will be used to support this broader assessment.


(1)

     Decision No 1350/2007/EC of the European Parliament and of the Council of 23 October 2007 establishing a

   second programme of Community action in the field of health (2008-13) (OJ L 301, 20.11.2007, p. 3–13).

(2)

     The external evaluation report is annexed, with executive summaries in English and French.

(3)

     COM(2007) 630 final, 23.10.2007.

(4)

     COM(2010) 2020 final, 3.3.2010.

(5) ‘Other’ includes actions signed and committed to by DG SANTE and CHAFEA, such as special indemnities to experts for their participation in and work for EU scientific committees, an administrative agreement with the Joint Research Centre, publications and various communication initiatives to promote the 2nd Health Programme, sub-delegations to Eurostat, etc.
(6)

     Directive 2011/24/EU of the European Parliament and of the Council of 9 March 2011 on the application of    patients’ rights in cross-border healthcare (OJ L 88, 4.4.2011, p. 45-65).

(7)

      http://www.orpha.net/consor/cgi-bin/index.php

(8)

      http://ec.europa.eu/chafea/documents/health/hiv-infopack_en.pdf

(9)

      http://ec.europa.eu/chafea/documents/health/nutrition-physical-activity-2014_en.pdf

(10)

      http://ec.europa.eu/chafea/documents/health/health-inequality-brochure_en.pdf

(11)

      http://ec.europa.eu/chafea/documents/health/mental-health_en.pdf


(12)

     The findings and conclusions of the mid-term evaluation were transmitted to the European Parliament, the Council, the European Economic and Social Committee and the Committee of Regions, as required by Decision No 1350/2007/EC, with staff working document SWD(2012) 83 final of 29 March 2012.

(13)

     Regulation (EU) No 282/2014 of the European Parliament and of the Council of 11 March 2014 on the establishment of a third Programme for the Union’s action in the field of health (2014-2020) and repealing Decision No 1350/2007/EC (OJ L 86, 21.3.2014, p. 1–13).

(14)

     The findings and conclusions of the final evaluation are discussed in detail in the Commission staff working document annexed to this report.

(15)

     The Commission reports annually to the European Parliament and to the Council on the implementation of the Health Programme (see http://ec.europa.eu/health/programme/policy/index_en.htm ).

(16)

      http://ec.europa.eu/chafea/projects/database.html .

(17)

      http://ec.europa.eu/chafea/publications/publications_for_health_programme.html .

(18)

     Meetings organised in cooperation with competent Member State authorities to provide journalists and other interested audiences with an opportunity to learn about EU health policy and a portfolio of relevant Health Programme actions in a given topic area.

(19)

      http://ec.europa.eu/chafea/health/EAHC_conference_2013_-_Transplantation_Blood_Transfusion.html .

(20)

      http://ec.europa.eu/chafea/health/hiv-athens-2014-presentations_en.html .

(21)

      http://ec.europa.eu/health/programme/events/ev_20120503_en.htm .

(22)

      http://ec.europa.eu/chafea/health/events_en.html .

(23)

     Investing in health (staff working document SWD(2013) 43 final, 22.2.2013)    
http://ec.europa.eu/health/strategy/docs/swd_investing_in_health.pdf .

(24)

   The Commission provides up to 80 % co-funding as a financial incentive for joint actions that involve at least 14 participating countries, of which at least four are Member States with a GNI per inhabitant below 90 % of the Union average, and for which 30 % of the budget is spent on involving such countries (Article 7(3) of Regulation (EU) No 282/2014).

(25)

     See at http://ec.europa.eu/priorities/index_en.htm