Annexes to COM(2024)49 - Ex post evaluation of Horizon 2020, the EU framework programme for research and innovation

Please note

This page contains a limited version of this dossier in the EU Monitor.

dossier COM(2024)49 - Ex post evaluation of Horizon 2020, the EU framework programme for research and innovation.
document COM(2024)49
date January 29, 2024
agreement. These changes helped accelerate the process to award grants, improved error rates and administrative expenditure that performed well against benchmarks. Key supporting factors mentioned were the new electronic grant management workflow and the elimination of the negotiation stage. However, the evaluation suggests that further tightening the time-to-grant target might not be necessary as it could inadvertently increase financial error risks.

Horizon 2020 marked a significant improvement in the efficiency of grant processing compared to its predecessor, FP7:

• 90% of grants were signed on time, a substantial increase from the 41% rate under FP7.

• the average time-to-grant period fell to 187 days (excluding ERC grants) from 313 days under FP7. This represents a considerable time cost saving for the EU administration and it benefited applicants, who were informed earlier and could start their projects at an earlier date.

Despite these advancements, the evaluation does not present an overall positive picture regarding the programme’s error rate. The European Court of Auditors pointed out that, particularly in operational expenditure and personnel costs, the level of error remains high and often avoidable.

Looking ahead, there is scope to improve the efficiency of the EU framework programme. Many stakeholders have indicated that participating in Horizon 2020 requires more effort than for other research and innovation funding programmes. This is significant given the programme’s relatively low success rate, as it means that a considerable share of the application cost represents a net loss to EU society. Any effective measure that reduces these costs has a strong potential to improve programme efficiency.

 


6. CONCLUSIONS AND LESSONS LEARNED

The evaluation concludes that Horizon 2020 significantly contributed to building an EU society and economy rooted in knowledge and innovation. By leveraging additional R&I funding, it supported not only the implementation of other EU policies but also played a crucial role in achieving and maintaining the European Research Area. Horizon 2020’s relevance to societal needs was evident in its rapid and flexible response to emergency situations such as the Ebola and COVID-19 outbreaks, and its influential role in global efforts to tackle climate change. The evaluation underscores the need to sustain action across multiple framework programmes to achieve these outcomes.

While Horizon 2020 met some of its targets or key performance indicators, it did not achieve all of them. This partial success can be attributed to the inherent nature of R&I investments, which often require a lengthy period to yield usable results. Many projects are still ongoing and there were also shortcomings in the initial setup of the programme’s indicators. Despite these challenges, the evaluation confirms that the results produced with Horizon 2020 funding are of high value.

The interim evaluation of Horizon 2020 led to some significant adjustments in the latter half of the programme. New measures to increase open science have borne fruit and the level of international participation was maintained. Further improvements, such as promoting women’s participation, better integrating social sciences and humanities and reducing the administrative burden have been carried forward and reinforced under Horizon Europe. In addition, the monitoring and evaluation framework has been revised to better track impact over time. The effectiveness of these measures will be closely examined in the interim evaluation of Horizon Europe.

This final evaluation of Horizon 2020 highlighted several key areas for further improvement, providing insights that will feed into future enhancements.

- Broadening participation. There is scope to broaden participation in the programme. It would involve engaging with non-traditional players from multiple sectors, scientific disciplines and countries. While national reforms of R&I systems can influence readiness for European-level project participation, the programme itself can be improved by greater simplification, visibility and accessibility.

- Further simplification needed. The programme can benefit from a targeted use of the two-stage application process, especially in areas with low success rates and a high volume of unsuccessful applicants. Extending the use of the Seal of Excellence certification scheme could also enable more applications to be reused for other programmes, reducing wasted effort. There is also further potential for simplification in extending the monitored use of lump-sum funding, as well as in improving outreach, information dissemination, and the user experience of programme tools.

- Dissemination, exploitation and deployment of results. The process of disseminating, exploiting and deploying project results has been uneven and requires more attention. Horizon Europe encourages applicants to give greater thought to the pathway to impact in their applications. Improvements are also needed to ensure the visibility, spread and practical use of project results to unlock broader economic and societal benefits.

- Supporting women in research and innovation. Despite efforts, it remains a challenge to achieve gender balance in research, entrepreneurship and innovation. Stronger measures are needed to support women researchers, entrepreneurs and innovators both Europe-wide or within the framework programme.

- Unlocking more synergies with other initiatives. Synergies with other EU, national and regional initiatives could be strengthened, particularly to support the uptake and use of project results. This includes better alignment to ensure the smooth operation of research infrastructures.


7. NEXT STEPS

The insights and key conclusions drawn from this final evaluation of Horizon 2020 are set to play a crucial role not only in shaping the ongoing implementation of Horizon Europe, but also in influencing policy development for future research and innovation initiatives. This will ensure that the lessons learned from Horizon 2020 are effectively integrated into current and future programmes to further improve their efficiency, relevance and impact on Europeans.


1 Under Article 32(4) of Regulation (EU) No 1291/2013 of the European Parliament and of the Council of 11 December 2013 establishing Horizon 2020 - the Framework Programme for Research and Innovation (2014-2020), complemented by Regulation 1290/2013 and by Council Decision 2013/743.

EN EN