Explanatory Memorandum to COM(2007)375 - Report on the Interim Evaluation of the Erasmus Mundus Programme 2004-2008

Please note

This page contains a limited version of this dossier in the EU Monitor.

dossier COM(2007)375 - Report on the Interim Evaluation of the Erasmus Mundus Programme 2004-2008.
source COM(2007)375 EN
date 02-07-2007
Important legal notice

|
52007DC0375

Report from the Commission to the Council, the European Parliament, the European Economic and Social Committee and the Committee of the Regions - Report on the Interim Evaluation of the Erasmus Mundus Programme 2004-2008 /* COM/2007/0375 final */


[afbeelding - zie origineel document] COMMISSION OF THE EUROPEAN COMMUNITIES

Contents

1.

Brussels, 2.7.2007


COM(2007) 375 final

REPORT FROM THE COMMISSION TO THE COUNCIL, THE EUROPEAN PARLIAMENT, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS

Report on the Interim Evaluation of the Erasmus Mundus Programme 2004-2008

REPORT FROM THE COMMISSION TO THE COUNCIL, THE EUROPEAN PARLIAMENT, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS

Report on the Interim Evaluation of the Erasmus Mundus Programme 2004-2008

TABLE OF CONTENTS

2.

1. Introduction 3


3.

2. Background to the external evaluation 3


4.

3. The external evaluation 3


5.

3.1. The terms of the evaluation 3


6.

3.2. Methodology 4


3.3. The evaluator’s findings 4

7.

3.3.1. Financial aspects 4


8.

3.3.2. European added value 5


9.

3.3.3. Relevance 5


10.

3.3.4. Effectiveness and impact 5


11.

3.3.5. Efficiency and cost-effectiveness 6


12.

3.3.6. Utility, added value and sustainability 7


13.

4. Main recommendations of the external evaluation and comments from the Commission 7


14.

4.1. Programme design 7


15.

4.2. Programme management 9


16.

4.3. Programme funding 10


5. The Commission’s conclusions 10

17.

Statistical Annexes on application and selection figures 12


18.

1. INTRODUCTION


THIS REPORT IS PRESENTED under Article 12 of the Decision 2317/2003/EC i of 5 December 2003 establishing the Erasmus Mundus Programme, which requires an interim evaluation of the programme to be carried out. It puts forward the Commission’s position on the main conclusions and recommendations of the Interim Evaluation of the Programme that can be obtained via the link: ec.europa.eu/dgs/education_culture/evalreports These conclusions and recommendations are based on extensive surveys of Erasmus Mundus participants and key stakeholders, the detailed results of which form an annex to the Interim Evaluation report.

19.

2. Background TO THE EXTERNAL EVALUATION


Erasmus Mundus is a co-operation and mobility programme in the field of higher education intended to promote the European Union as a centre of excellence in learning around the world. It aims to support the development of top-quality European Masters Courses and to enhance the visibility and attractiveness of European higher education in third countries. The programme has, as its strategic aims, to improve the quality of higher education in Europe and to promote intercultural understanding through co-operation with third countries.

The specific aims of the programme are to: promote quality and excellence in European higher education; encourage the incoming mobility of third-country graduate students and scholars; foster structured co-operation with third-country higher education institutions; and improve the profile, visibility and accessibility of European higher education in the world.

The programme has four main Actions. These are: Action 1 - Erasmus Mundus Masters Courses, comprising integrated courses at masters level offered by at least three universities in three different European countries; Action 2 - Erasmus Mundus scholarships for students and scholars from third countries; Action 3 - Partnerships with higher education institutions in third countries, comprising scholarships for students and scholars from EU countries for mobility towards third countries; Action 4 - Projects to enhance the worldwide attractiveness of European higher education.

20.

3. THE EXTERNAL EVALUATION


21.

3.1. The terms of the evaluation


Following a call for tender i, the Centre for Strategy & Evaluation Services (CSES) was selected to carry out the evaluation. The scope of the interim evaluation was the period 2004-2006, during which a number of calls for proposals have taken place to implement the programme. While Erasmus Mundus Masters Courses, scholarships and “attractiveness projects” (Actions 1, 2 and 4) commenced in the academic year 2004-2005, Partnerships (Action 3) began one year later in 2005-2006.

The aims of the interim evaluation were to: address the relevance and utility, efficiency, effectiveness, sustainability and Community Value Added of the programme; and provide the Commission with recommendations as to how the programme’s intervention logic, objectives, design, implementation arrangements, results and impact can be further improved.

22.

3.2. Methodology


The methodology included desk research of relevant literature; a survey of institutions participating in Erasmus Mundus Masters Courses, Partnerships and attractiveness projects (Actions 1, 3 and 4), and of participating third-country and EU students and scholars; and interviews with key stakeholders, including the Commission, the Education, Audiovisual & Culture Executive Agency (EACEA), Erasmus Mundus National Structures and participants in the four Actions of the programme.

3.3. The evaluator’s findings

Overall, the Erasmus Mundus programme 2004-2008 appears to have made a very positive start. The programme has generated genuine enthusiasm amongst students and higher education institutions (“HEIs”) alike, suggesting high relevance to identified needs. The programme was perceived by HEIs as adding value in a number of ways, for example by promoting the development of joint, double and multiple degree awards between HEIs in different countries, in line with the aims of the “Bologna process” of strengthening the European dimension in education and promoting increased mobility.

Erasmus Mundus has also begun to make a contribution to the promotion of academic excellence in European higher education, in particular by encouraging European HEIs to foster co-operation and joint working with other HEIs regarded as “world-class” in particular subject disciplines. Likewise, from the perspective of students, a wide range of benefits were identified, including the personal development benefits that arise from exposure to new cultures and languages and the academic benefits of studying on a Masters Course which demonstrates academic excellence.

Looking ahead, participation in Erasmus Mundus was also viewed as potentially bringing benefits for students in terms of their future career development. However, given that the programme only commenced in 2004, its impact on the employment prospects of potential students will need to be assessed through longitudinal studies of Erasmus Mundus graduate destinations (an initial tracking study will shortly be launched in this regard).

23.

3.3.1. Financial aspects


The Erasmus Mundus programme was allocated a budget of 230m euros for the period 2004-2008. Supplementary financing of 57.3m euros was made available in the years 2005-2007 through the “Asian Windows” as well as 8.8m euros in the year 2007 through the “ACP Window” and the “Western Balkans Window”. These are financial envelopes to fund additional scholarships for students from specific countries which have been allocated through the EU’s external aid budget. Consequently, a total budget of 296.1m euros is available for the programming period 2004-2008.

The overall financial envelope for the programme allowed the funding of a number of high-quality courses and scholarships that was in line with initial expectations.[3] However, demand for both courses and scholarships increased throughout the period under review and was of a sufficiently high level to justify additional funds in the future. For example, for Erasmus Mundus Masters Courses and attractiveness projects (Actions 1 and 4) approximately 1 in 7 applications were successful between 2004 and 2006, while for Partnerships (Action 3) approximately two-thirds of applications were successful. In addition, in 2007 approximately 1 in 7 applications for scholarships (Action 2) were successful, which suggests that competition among third-country students for these scholarships is high. Moreover, the easy absorption of the “Windows” suggests that a bigger financial envelope could have been absorbed without difficulty and with a significant increase in impact for the programme.

24.

3.3.2. European added value


One of the core elements of the Erasmus Mundus programme is the requirement to include a strong transnational dimension to Masters Courses, with mobility in a minimum of two different EU countries being an integral component. Participants and stakeholders therefore rated Erasmus Mundus highly in respect of Community Value Added. The close coherence between Erasmus Mundus and the objectives of the Bologna process (for example, through the promotion of the European dimension in education, mobility, joint degrees and European co-operation on quality assurance) also suggests Community Value Added.

3.3.3. Relevance i

The policy context has not changed radically since the proposal was drawn up for the 2004-2008 programming period. The intervention logic therefore remains relevant both from a policy perspective (strengthening co-operation with third countries in the field of higher education, fostering intercultural dialogue, promoting the development of integrated courses leading to the award of joint, double or multiple degrees at European level in line with the aims of the Bologna process) and in meeting the identified needs of programme beneficiaries (higher education institutions and students and scholars from both Europe and third countries).

There are strong linkages between Erasmus Mundus and the Lisbon strategy, notably the Education and Training 2010 agenda, and the Bologna process, which emphasises the need to open up education and training systems to the wider world as part of the Community’s response to the challenges and opportunities presented by globalisation. Europe must compete in an environment where the number of international students is growing rapidly and where competition for these students is intensifying.

3.3.4. Effectiveness and impact i

Outcomes to the end of 2006 were as follows: 80 Erasmus Mundus Masters Courses (Action 1); 2325 scholarships for incoming third country students (Action 2); 19 Partnerships (Action 3); 23 attractiveness projects (Action 4).[6] Participation rates in the programme so far have been broadly in line with expectations, with the exception of Partnerships (Action 3), where participation was lower than anticipated.

Erasmus Mundus has made some progress in beginning to develop more structured co-operation in higher education between the EU and third countries, although the low participation rates in Partnerships (Action 3) suggest that more still needs to be done in this area. In terms of improving the accessibility of European higher education, in particular by enabling highly qualified graduates and scholars from all over the world to study and/or teach in the EU, the scholarship scheme (Action 2) has greatly facilitated access to, and increased the attractiveness of, European higher education for high calibre students from third countries.

With regard to the promotion of intercultural understanding, Erasmus Mundus has been successful in encouraging cultural exchange of experiences. This process has been assisted by the fact that there was a very good distribution of nationalities among third-country scholarship holders. Moreover, students from third countries identified cultural and linguistic benefits as being more important than the benefit to their future career prospects. However, students from third countries have to date benefited more from the programme than EU students due to the more limited financing scope for the mobility of EU students. The experience of students - academically, culturally and linguistically - is much richer in Erasmus Mundus courses where there is a good balance between European (including host country) and international students, as opposed to courses that are dominated by non-European students.

Erasmus Mundus has also had a positive impact on those EU countries where there was previously no legal framework for the accreditation of joint, double or multiple degrees involving partners in different EU countries. Even where joint degree accreditation problems remain, Erasmus Mundus Masters Courses have implemented effective double or multiple degree structures which have permitted genuine integration and co-operative study to take place. Joint degrees also help make the EU labour market more transparent and accessible to European students.

Less positive was the fact that certain European countries were underrepresented in terms of the number of projects in which their institutions participated, an imbalance that the Commission is looking for ways to address. The Erasmus Mundus Masters Courses cover practically all academic disciplines, although there is a clear preponderance of engineering and natural science courses.

3.3.5. Efficiency and cost-effectiveness i

The Commission and the EACEA were regarded by the National Structures and HEIs running Erasmus Mundus Masters Courses as having done a very good job in overseeing the programme’s implementation to date. Communication and information flows in particular were regarded highly favourably as regular and informative. Responses to queries from HEIs have normally been dealt with promptly and useful feedback and comments have been provided to consortia on their reports. The overall high levels of satisfaction in respect of programme management by students and HEIs alike suggests that the programme has been managed both efficiently and effectively by the Commission and the EACEA.

The use and the level of unit costs and lump sums applied for the implementation of the programme were largely considered cost-effective, helping the programme to reach its aims at the lowest possible costs. However, the evaluation suggests that universities participating in Erasmus Mundus Masters Courses and European students are under-funded in the current programme.[8]

3.3.6. Utility, added value and sustainability i

Quite a large number of Erasmus Mundus Masters Courses supported through the programme were pre-existing courses that have been modified to fit into the Erasmus Mundus framework. The programme seems to have encouraged HEIs to modify their existing high-quality courses through co-operation and joint curriculum development with HEIs in other European countries offering the same subject discipline and able to meet the requisite academic excellence criteria at European/international level.

The research suggests that most of these courses could not have continued in their present form without funding from the Erasmus Mundus programme. While this suggests low levels of sustainability, looked at from another perspective it suggests that European money is being used to promote activities that would not otherwise be able to take place on the same basis.

Another issue is whether students and scholars (particularly those from third countries) would have participated in the programme without financial support. In this context, 95% of third-country students stated that they could not have participated in the programme without the scholarship. This suggests high levels of additionality.

25.

4. MAIN RECOMMENDATIONS OF THE EXTERNAL EVALUATION AND COMMENTS FROM THE COMMISSION


The main recommendations of the evaluator are presented in bold, while the Commission’s answer is in italics.

26.

4.1. Programme design


Recommendation 1

Scholarships should be awarded to EU students to participate in the Erasmus Mundus programme on a competitive basis. There is a need to ensure that EU students participate on a more equal footing with their counterparts from third countries.

The Commission agrees with this recommendation, as a way of facilitating intra-EU mobility for EU students and encouraging stronger participation by students from EU countries. It will examine possible measures to improve the position of EU students, while also striving to ensure complementarity with the Erasmus programme.

27.

Recommendation 2


The Erasmus Mundus programme should - finance permitting - be extended to the PhD level both at the level of courses and in respect of scholarships. Issues around quality assurance in respect of PhD programmes will need to be carefully thought through. The French co-tutelle model should serve as a model in this regard. Particular care should be taken to avoid duplication with Marie Curie research scholarships.

The Commission agrees with this recommendation as a way of enabling Europe to retain excellent students and researchers from third countries as well as of creating links between higher education and research. It will examine the “co-tutelle” model when deciding on future measures, while also taking care to establish synergies with Marie Curie research scholarships and networks.

28.

Recommendation 3


Erasmus Mundus Masters Courses should be encouraged not only to develop co-operation with higher education institutions in third countries but to enable these institutions to become fully fledged partners in the course itself. This could be achieved by bringing Masters Courses and Partnerships (Actions 1 and 3) together under a single umbrella.

The Commission agrees in principle with this recommendation and will examine appropriate measures for the future. However, quality assurance is a complex exercise when it comes to HEIs located in third countries. It is important to ensure that the wishes of participating European HEIs and the structure of their academic programmes are fully respected.

29.

Recommendation 4


The Atlantis and the EU-Canada Cooperation Programme (and likewise pilot projects with Japan, Australia and New Zealand) should be combined with the Erasmus Mundus programme so as to improve the coherence of the Commission’s approach to strengthening co-operation in the field of higher education between the EU and third countries. From a promotional and awareness-raising perspective, there would be advantages in using Erasmus Mundus as the single brand through which institutional co-operation and student and scholar mobility between the EU and third countries is promoted. This would also have a significant positive effect on the visibility of the programme and the coherence of Community activities in the field.

The Commission considers this recommendation interesting. However, the EU-US Cooperation Programme (Atlantis), which was recently renewed, has a wider remit than Erasmus Mundus, with joint masters courses being only one part of the programme. Combining all Community higher education activities with an external dimension into a single integrated programme could, however, be an option after 2013.

30.

Recommendation 5


Consideration should be given to providing Erasmus Mundus students with placement opportunities on a more systematic basis. This would also contribute significantly to raising the profile of the programme among employers and wider stakeholders. However, undertaking a placement should not be made compulsory since this would not necessarily be appropriate in the case of all Erasmus Mundus Masters Courses.

The Commission welcomes this recommendation and will encourage Erasmus Mundus Masters Courses to offer placements to their students, where appropriate. A flexible approach to this issue, as suggested by the evaluators, seems adequate.

31.

4.2. Programme management


Recommendation 6

The National Structures should continue to be financed largely by the Member States in order to avoid a full EU Agency structure at national level (which does not appear to enjoy support amongst National Structures). Nevertheless, some EU financing should be made available to help co-finance promotional activities by the National Structures since their workload is likely to increase significantly in the next programming period due to the projected increase in the programme budget. The Commission should make provision for restricted calls for proposals on a thematic basis to enable the National Structures to finance specific initiatives they wish to support, such as marketing and promotional activities etc. Consideration could be given in the new programming period to giving the National Structures a formal role in contributing to the monitoring of Erasmus Mundus Masters Courses.

The Commission agrees with this recommendation. The role of National Structures in carrying out specific information, promotion or dissemination activities, as well as monitoring of projects, should be reinforced.

32.

Recommendation 7


There is a need for the European Commission to turn its attention to quality assurance once Erasmus Mundus-branded Masters Courses are actually up and running. Self-evaluation should remain the fundamental starting point for ensuring continuous course quality. In addition, a representative sample of Erasmus Mundus Masters Courses should, over the lifetime of each programming period, be subject to external quality assessment. This work could be carried out by external quality assurance bodies that have had previous experience in assessing the quality of academic content and the integration of courses (preferably of courses delivered on a transnational basis).

The Commission agrees with the need to ensure the quality of Erasmus Mundus Masters Courses and is already tackling this issue under the current programme. In 2007, the Commission will elaborate - in co-operation with external experts in the field - guidelines on good practice in Erasmus Mundus Masters Courses. Following a self-assessment exercise, a sample of courses will be visited by external experts, within the context of a peer-review exercise, with a view to establishing indicators of good practice and disseminating these to current and to potential future beneficiaries.

33.

Recommendation 8


The comitology principle for Erasmus Mundus scholarships (Action 2) should be discontinued in order to speed up the finalisation of the selection process. This could accelerate the scholarship award decision by as much as 6-8 weeks.

The Commission shares the concern of the evaluators about the current comitology procedure for selection decisions on scholarships, since these decisions are taken purely by universities according to criteria based on academic merit. The Commission will examine this issue when suggesting the design for a future programme.

34.

4.3. Programme funding


Recommendation 9

Looking to the next programming period, the scholarship level for third-country students should not be reduced from the current level of 21,000 euros per year. However, the Commission should also continue to monitor the scholarship level of other renowned scholarship schemes, such as the Fulbright, Chevening and DAAD (German Academic Exchange) scholarships. A universal scholarship amount should continue to be given with no differentiation either on the basis of where a given student decides to study or on the grounds of their country of origin. This is the only equitable approach and other approaches do not seem workable. Common tuition fees determined by individual Masters consortia should be retained in respect of Erasmus Mundus Masters Courses.

The Commission agrees with this recommendation in order to maintain globally competitive levels of funding for Erasmus Mundus scholarships. As regards the common tuition fee, the Commission believes that this is an essential element of an integrated study programme and intends to continue supporting common tuition fees as an obligatory element of joint programmes.

35.

Recommendation 10


In the next programming period, the amount of finance allocated to each Erasmus Mundus Master consortium should be increased to reflect the true cost of administering an integrated Masters Course on a cross-border basis.

The Commission itself has seen, through monitoring of the programme, that the current annual grant of 15,000 euros given to universities generally underestimates the costs of joint programmes. It will take this recommendation into account when planning for the next phase of the programme, in full respect of the Financial Regulation and its Implementing Rules.

5. THE COMMISSION’S CONCLUSIONS

The Commission shares the overall assessment of the evaluator that the Programme has provided an important contribution to the internationalisation of European higher education. Erasmus Mundus has been a relevant and efficient tool for European HEIs in helping them to find a response to globalisation. The results of this interim evaluation show that the programme is meeting its political and operational objectives, and the objectives of Article 149 of the Treaty.

Indeed, 323 HEIs in Europe and third countries and 2,325 third-country students participated in the programme from 2004 to 2006, underlining their overall satisfaction - and even enthusiasm - for the programme. The Commission, when proposing the future Erasmus Mundus programme, will pay due respect to the criticism voiced that the programme has been less successful with respect to European students.

The Commission will take the results of this interim evaluation into account when proposing the new Erasmus Mundus programme for beyond its current programming period of 2004-2008. It will base its proposal for the new Erasmus Mundus programme which will be presented in June 2007 on the recommendations made by this evaluation, which happen to coincide with findings of surveys which were conducted outside the scope of this evaluation but during the same timeframe.

36.

STATISTICAL ANNEXES ON APPLICATION AND SELECTION FIGURES


HEIs participating in Erasmus Mundus Masters Courses (2004-2006)

EU + EFTA/EEA Countries HEI instances of participation in EM Masters Courses applying to EM i HEI instances of participation in EM Masters Courses selected under EM i HEIs participating in EM Masters Courses selected under EM Success rate in %

Austria 26.

Belgium 14.

Cyprus 0.

Czech Rep. 18.

Denmark 22.

Estonia 15.

Finland 17.

France 13.

Germany 16.

Greece 11.

Hungary 14.

Ireland 15.

Italy 11.

Latvia 0.

Lithuania 0.

Luxembourg 16.

Malta 6.

Netherlands 21.

Poland 10.

Portugal 15.

Slovakia 10.

Slovenia 11.

Spain 11.

Sweden 20.

UK 18.

Iceland 0.

Liechtenstein 0.

Norway 23.

Total 14.

These figures include double or multiple participation by the same HEIs in various applications.

HEIs participating in Erasmus Mundus “Attractiveness Projects” (Action 4) (2004-2006)

EU + EFTA/EEA Countries HEI instances of participation in EM Action 4 projects applying to EM i HEI instances of participation in EM Action 4 projects selected under EM i HEIs participating in EM Action 4 projects selected under EM Success rate in %

Austria 12.

Belgium 9.

Cyprus 33.

Czech Rep. 10.

Denmark 26.

Estonia 30.

Finland 27.

France 14.

Germany 13.

Greece 6.

Hungary 8.

Ireland 33.

Italy 7.

Latvia 25.

Lithuania 15.

Luxembourg 0.

Malta 20.

Netherlands 25.

Poland 11.

Portugal 11.

Slovakia 10.

Slovenia 0.

Spain 14.

Sweden 17.

UK 13.

Iceland 0.

Liechtenstein 0.

Norway 43.

Total 14.

These figures include double or multiple participation by the same HEIs in various applications.

37.

Student scholarships


Number of student applications received Erasmus Mundus scholarships awarded “Window” scholarships awarded Total number of scholarships awarded Success rate in %

2004- n.a. n.a.

2005- 26.

2006- 25.

2007- 12 14.

2008-09 i n.a. n.a.

Total 21 19.