Legal provisions of COM(2011)653 - Interim Evaluation of External Cooperation Agreements in Higher Education, Training and Youth with the USA and Canada - Main contents
Please note
This page contains a limited version of this dossier in the EU Monitor.
| dossier | COM(2011)653 - Interim Evaluation of External Cooperation Agreements in Higher Education, Training and Youth with the USA and Canada. |
|---|---|
| document | COM(2011)653  |
| date | October 21, 2011 |
|
52011DC0653
/* COM/2011/0653 final */ REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS on the Interim Evaluation of External Cooperation Agreements in Higher Education, Training and Youth with the United States of America and Canada
REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS
on the Interim Evaluation of External Cooperation Agreements in Higher Education, Training and Youth with the United States of America and Canada
INTRODUCTION
This Report has been established pursuant to Article 7 of the Agreements between the European Community and the United States of America signed on 4 December 2006 and between the European Community and the Government of Canada signed on 18 December 2006 in the areas of higher education, training and youth. It puts forward the Commission position on the main conclusions and recommendations of the interim evaluation of these two Agreements. This evaluation can be found at:
http://ec.europa.eu/dgs/education_culture/evalreports/index_en.htm.
While recognising the evaluation’s results, unfortunately, due to budget constraints, both the U.S. and Canadian authorities decided to cancel all current and future calls for bilateral projects between now and 2013 when both Agreements will come to an end. In its comments to the evaluators’ recommendations in section 5, the Commission takes due account of this unexpected turn of events.
THE COOPERATION AGREEMENTS
Aims
Both Agreements serve educational, people-to-people and public diplomacy purposes. They contribute to quality education and training in the European Union, through fostering cooperation with the United States and Canada. The Agreements seek to impact in two areas:
1. promotion of intercultural dialogue and mutual understanding; and
2. development of skills and competences relevant to the knowledge-based economy.
Instruments of Intervention
The main actions carried out under the Agreements are multi-institutional transnational partnerships that
- involve higher education and/or training institutions from both North America and the EU,
- cooperate to develop joint curricula at Bachelor/Master level including joint/double degrees and
- promote transatlantic mobility of both students and faculty.
Actions specifically undertaken in the framework of the EU-US Agreement are:
3. the ATLANTIS programme with three sub-actions: (i) Transatlantic Degree Projects (TD); (ii) Excellence in Mobility Projects (EIM); and (iii) Policy-oriented Measures (POM);
4. the Schuman-Fulbright grant scheme;
5. the OCEANS alumni association; and
6. technical support measures, including annual Policy Forums and studies.
Actions specifically carried out under the EU-Canada Agreement are:
7. the EU-Canada programme consisting of higher education and training actions, notably Transatlantic Exchange Partnerships (TEP) and Transatlantic Degree Partnerships (TDP);
8. youth actions, in particular youth roundtables; and
9. complementary actions, notably the EU-Canada Study Tour and Internship Programmes and the OCEANS alumni association.
More details about these instruments and background documents can be obtained from the Commission’s websites.
Funding and Outputs
As a general rule, the European Union provides funding for the direct use of EU institutions, scholars and students while the United States of America and the Government of Canada provide funding for the direct use of institutions, scholars and students from their respective countries.
The European Union's budget allocation for the ATLANTIS programme was EUR 23.2 million for the 2006-10 period and for the EU-Canada programme was EUR 5.1 million for the 2007-10 period. Programme financing is done on the basis of a matching-funds principle between the parties to each Agreement, meaning that the EU budgets are in the end doubled by matching funds provided by the respective US and Canadian authorities.
The ATLANTIS programme has permitted so far a total of 5 034 individuals and 317 institutions to participate in 67 mobility projects .
The EU-Canada programme has permitted so far a total of 2 804 individuals and 195 institutions to participate in 37 mobility projects.
The Schuman-Fulbright grant scheme is co-funded by the EU with an amount of EUR 320 000 per year and has had so far a total of 83 participants.
THE EVALUATION
The contract for the interim evaluation of the EU-US and EU-Canada Agreements was awarded to the company ECOTEC/ECORYS in March 2010. The evaluation was carried out between March 2010 and January 2011. The Commission received the Final Evaluation Report on 15 June 2011.
With regard to the methodology, the evaluators gathered the data for their assessment through discussions with North American stakeholders in Washington, DC and Ottawa; discussions with European Commission officials and staff from the Education, Audiovisual and Culture Executive Agency (EACEA) and the Belgian-Luxembourg Fulbright Commission in Brussels; in-depth interviews with 95 project partners and coordinators (and students) from 20 thematic project cases including visits to each of the projects concerned; an online survey with project partners and coordinators, which generated 209 responses; online surveys of students and Schuman-Fulbright grantees, which generated 53 and 57 responses respectively; a telephone survey of 5 rejected applicants; and a review of relevant documentation.
Results of the EVALUATION
The results of the evaluation are generally very positive, having analysed the degree of relevance of the Agreements to needs; the extent to which they overlap, complement or conflict with other policies and actions; the effectiveness of the Agreements in meeting their intended objectives and effects, and their added value; their efficiency and cost-effectiveness, and the effectiveness of management arrangements and other operational issues such as dissemination and exploitation of project results; and the extent to which the activities that have been supported are sustainable.
Relevance to Needs
The evaluators concluded that the Agreements are highly relevant in responding to a set of needs that are important and increasingly significant: globalisation and the parallel process of internationalisation in education. The Agreements' objectives are found
- to be broad-ranging and flexible, enabling governments to pursue a variety of political and economic goals;
- to provide an on-going platform for discussion at a time of increasing complexity and competition, protecting activity which, experience elsewhere shows, can be very fragile;
- to provide opportunities for new players to enter the international education arena; and
- to offer the chance for institutions to engage in the most sophisticated and leading-edge forms of international partnership in the form of multi-institutional cooperation and joint/double degrees.
Complementarity with other Policies and Actions
The evaluators found that the programmes do not overlap or conflict with the wide range of EU initiatives in the fields of international cooperation, mobility, education and training with which the Agreements share some similar features. On the contrary, the Agreements are unique in a number of ways, especially due to their nature as international bilateral Agreements and the benefits derived from this form of institutionalised cooperation under public international law. While the activities supported by the Agreements are comparatively small in size in the context of the volume of international cooperation routinely undertaken in this sector between the EU, the US and Canada, they are notwithstanding offering a wide range of opportunities. Only when it comes to the field of youth (Canada only), the evaluators concluded a lack of implementation which made it difficult to assess complementarity.
Effectiveness
The evaluators found that the Agreements have made contributions in respect of all their intended objectives, but have made less progress than anticipated in respect of providing opportunities for policy discussion and exchange, and in Canada, in the youth field. The Agreements have substantially benefitted the students participating in the programmes in their personal and professional development (soft skills, language proficiency, intercultural competences and competences useful in the global labour market and international work environments). The Agreements were found to be of major benefit also to the staff and professionals as well as to the institutions involved in the projects, in particular in terms of setting up and maintaining international networks, increasing profile and reputation, building capacity to operate internationally and creating 'change agents' who can cascade the positive outcomes throughout the respective institutions.
Added Value
The evaluation concluded that the bilateral funding aspect of the Agreements had a significant added value at a strategic level and that the Agreements fill a clear 'gap in the market' for the funding of international cooperation between the EU and the US or Canada. Most partners and coordinators believe their projects to be of higher quality, more useful and more worthwhile as compared to other international cooperation programmes in the field. The Agreements were found to stand as models for other industrialised countries, not the least due to the fact that they champion high-quality student mobility, promote the use of innovative approaches to curriculum development, transparency and portability and that they provide a means to shape the nature of international cooperation.
Efficiency
The evaluators found that while more data is needed to make a full assessment of the efficiency of the programmes, in comparison to e.g. the Erasmus Mundus programme and as a result of the matching funds principle, the average EU budget per exchanged student was significantly lower under the Agreements, speaking for the cost-efficiency of the Agreements. However, the extremely small budget available in the EU-Canada case presents a significant handicap in fulfilling all of the Agreements' objectives.
Sustainability
The evaluators concluded that there is a strong continuation of partnerships formed under the Agreements: half of the project partners and coordinators maintain contacts after completing their projects, and three quarters of the Schuman-Fulbright grantees continue to cooperate with colleagues from their host institutions, usually in research activities. However, at project level, while coordinators expect to continue cooperation owing to their significant personal investment into the project, due to a current constraint on institutions' resources, a full continuation of activities after a project's life-cycle is uncommon, and businesses are still a rare source of funding, especially amongst EU participants.
Promotion, Dissemination and Exploitation
The importance of achieving wider effects beyond the immediate projects themselves was clearly pointed out by the evaluators. However, certain obstacles in achieving this result were identified, most notably as to what concerns a presumed lack of visibility and/or renown of the programmes (with the exception of the Schuman-Fulbright grant scheme), as well as a clear brand name (Canada only). The evaluators further concluded that the publicity for the Agreements and the dissemination of the results could be improved. They also stated that the annual Project Directors' Conference should be maintained as an important forum offering opportunities for cross-fertilisation between different projects and institutions. The vertical communication of the results of the POMs to stakeholders should be improved. The potential benefits of the OCEANS alumni association in respect of promotion and dissemination were accentuated by the evaluators. The potential for exploitation of results within institutions was evaluated as high, while the evaluators already took into account the strong degree of exploitation between institutions.
Recommendations
The evaluators’ recommendations (in italics ) are outlined below, together with the comments of the Commission.
Relevance
10. The Agreements remain highly relevant to needs in HE, and the adoption of joint/double degrees in 2006 shows its ability to remain at the leading edge of developments. The non-implementation of these activities within the EU-Canada Agreement means that the Agreement's full potential is not being realised. Priority should be given to implementing these aspects of the Agreement in the remaining period to 2013.
The Commission and its Canadian counterparts took the necessary steps to encourage and implement joint/double degree projects in the bilateral call in 2010. As a consequence, one grant was given to a Transatlantic Degree Partnership in the area of forestry.
11. The Agreements are well tuned to the needs of HE, but less so to the needs of VET. Greater attention needs to be given to how to adjust the Agreements to better fit with the needs of the sector; promoting the Agreements more to the VET sector will not, alone, ensure greater interest and take-up.
The Commission has repeatedly discussed with its transatlantic counterparts ways to better cater for the needs of the VET sector through the annual calls for proposals. These discussions have led to the ease of some of the programme requirements for VET institutions.
12. Quality is taken for granted by participants and programme administrators. The Erasmus Mundus programme has sought to develop quality as part of the programme's brand value which is developed and ensured by the participants themselves; a similar approach is recommended for projects under the Agreements, especially in respect of joint/double degrees.
While agreeing with the tenor of this recommendation, the Commission points out that it is based on a fairly small and thus not fully representative sample of students’ feedback. The quality of projects and their results/outputs is, in general, very high. However, the Commission will look into how to further advance the quality of projects implemented under both Agreements. As a first step, it will make sure, through the EACEA, that all partners involved in bilateral projects are aware and make use of the Erasmus Mundus quality assurance handbook and self-assessment tool.
Effectiveness
13. Canada lacks a means for regular discussion and exchange of experience on policy issues. Whilst the division of responsibilities between federal and provincial governments may make it difficult to set up a mechanism similar to the US one, renewed efforts should be made to realise the goal of holding regular events on issues of mutual interest.
The Commission takes note of this recommendation . Through various channels, the Commission has been exploring ways of entering into a meaningful and mutually beneficial policy dialogue with Canada in the areas of education, training and youth. This has been effective so far at the level of pan-Canadian associations in the areas of higher education, vocational education and training as well as international education. The Commission will continue its efforts targeting in particular the federal and provincial levels in Canada.
14. The Agreements have made contributions to EU objectives in important areas, but have made less progress than anticipated in respect of providing opportunities for policy discussion and exchange, and, in Canada, in the youth field. Both these areas should be made a strategic priority in the latter half of the life of the Agreements. In Canada, a swift decision should be taken as to what extent the youth field is a strategic priority which deserves to be implemented during the last years of the current Agreement. In the event of continuing inactivity, EU budget allocated to this area should be re-allocated to other parts of the programme in order to prevent waste of resources.
Whilst policy discussion and exchange are less developed yet with Canada (see comments to recommendation (4) ), the Commission and the US Department of Education have already organised two Education Policy Forums: in 2009 on higher education reforms and university-business cooperation, and in 2010 on quality in higher education, schools and teacher development and skills for tomorrow. In 2011, another Forum is planned discussing the 21st-century student, qualifications frameworks and student learning.
With regards to the youth field and Canada, the Commission and its Canadian interlocutors have taken steps to implement a series of actions. In light of this recent development, a re-allocation of funds seems no longer necessary.
15. Stronger linkages should be explored under the EU-US Agreement between the Policy Forums and the Policy Oriented Measures to better realise the potential synergies, and to ensure that the findings from the POMs are considered at EU/national level.
The Commission agrees with this recommendation and is looking into concrete ways of increasing synergies between, and disseminating more widely the results of, both the Policy Forums and POMs.
16. The benefits of participation need to be made more widely available to less experienced and smaller institutions, especially in the VET field. Greater promotion should take place to target these institutions, e.g. through the development of a good practice guide targeted at the VET sector. Consideration should be given to putting in place a mechanism to assist inexperienced projects to undertake a project development phase. Furthermore, certain under-represented institutions could be given priority under the yearly calls for proposals, and efforts should be made to identify and support previously successful participants from under-represented institutions to become champions to promote participation in their sectors. For VET, consideration should be given to preparing a brochure of successful VET projects to target this sector specifically.
The Commission partially agrees with this recommendation . Efforts were made to facilitate access of VET institutions under past annual calls for proposals (see comments to recommendation (2) ): these calls made provisions for supporting a diverse range of institutions or for new and/or not-yet funded institutions to be selected on a priority basis.
The Commission points out, however, that all applicants need to be treated equally which means that assistance mechanisms may not be geared towards one kind of applicants only. The EACEA put in place several support tools for potential applicants, notably multiannual programme guides, FAQs, project compendia and functional mailboxes where questions that may have arisen during the application phase could be addressed to. In addition, the Commission launched EVE, the single information point on European education, training, culture and youth projects showcasing thousands of projects and their results, notably from the two Agreements. This tool for the dissemination and exploitation of project results is open to the wider world and could be used as a source of inspiration for less experienced or smaller institutions.
17. Four years after the renewed Agreement [with Canada] was launched there are very few concrete outcomes to show in the youth field, and very little momentum has been built up. Greater priority needs to be given to action in the field to make it a reality.
The Commission only partially agrees with this recommendation . Following a slow start, the youth field has gained momentum since 2009. See comments to recommendation (5) .
Efficiency
18. More structured communication processes on information needs and monitoring should be put in place, for instance by making better use of the inter-service meetings between the Commission and the EACEA.
The Commission agrees with this recommendation and will look into organising more regularly inter-service meetings with the EACEA.
19. More use should be made of up-to-date reporting tools, or at least standardised formats, to improve the efficiency for the project partners and coordinators, as well as for the EACEA in gathering information/data. It is recommended that a standard format be put into place for registering students and potential alumni at the moment of selection.
The EACEA has already started to review its reporting tools. In particular, a new web-based mobility database should be ready for roll-out in early 2012. It is anticipated that this new tool will specifically improve the tracking of students and alumni.
20. A structured meeting cycle between the European Commission (DG EAC) and the Belgium/Luxemburg Fulbright Commission should be put in place, to exchange qualitative information on the Schuman-Fulbright grant scheme and the (possible) synergy with the other actions under the Agreement with the US. In particular, the EC should be provided with regular qualitative information on the grantees, such as their geographical spread and their subjects.
A format for providing DG EAC with quantitative data on the grantees should be discussed and agreed.
The Commission agrees with this recommendation . Concrete measures will be initiated to obtain qualitative information on the Schuman-Fulbright grant scheme in a structured and regular way in view of creating synergies with other actions implemented under the Agreement with the US and other EU programmes.
21. A pragmatic and feasible approach should be developed to monitor the (real) inputs and outputs of the projects, in order to make cost-effectiveness visible, and a set of simple criteria should be formulated for assessing cost-effectiveness more qualitatively.
The Commission takes note of this recommendation , but also stresses the evaluators’ finding that “there is a good prima facie case that the Agreements do not give cause for concern with regards to cost-effectiveness”. Given the relatively small budgets allocated to both Agreements and individual projects, it may in fact be more cost-effective to develop such a pragmatic and feasible approach in the framework of a larger EU programme and then apply it mutatis mutandis to bilateral projects. It should be noted that the EACEA has put in place a desk and field monitoring system to assess the project implementation against the plans outlined in the agreed work programmes.
22. An inventory should be made of the most common (legal) obstacles and applied solutions and disseminated to all new applicants.
The Commission agrees with the gist of this recommendation , but points out that it already published in 2008 a good practice guide offering practical tips and tricks for designing and implementing a successful bilateral project. Furthermore, in the framework of the 2007 POM project entitled “Transatlantic Degree Programs (TDP) Inventory project”, a publication was developed addressing in particular the challenges and opportunities in developing joint and double degree programmes. This publication was distributed free-of-charge to all project promoters and partners at the Project Directors Conference in Berlin in October 2010. The Commission will look into how to best complement and consolidate the available information and disseminate it as widely as possible to interested audiences.
Sustainability
23. Maintain the principle of own investments by the participating institutions, to maintain the commitment of parties involved during and after financing.
The Commission has no intention to modify this principle.
24. Encourage more involvement and investment by partners outside the educational sector.
Since the inception of the programmes and in order to improve relations between education and training organisations and the world of work, the Commission has encouraged consortia to associate organisations such as enterprises, trade unions, industry and business groups, NGOs, publishers, government departments, chambers of commerce and research institutes to the project work. Such organisations may among others provide internships, offer professional advice and expertise, and help a project attain the national and international visibility necessary for it to succeed beyond the funding.
25. Create the possibility for innovative ‘continuation’ applications besides the EIM action, with the requirement that new partners should be included – preferably from countries less represented in the Agreements.
While the Commission takes note of this recommendation, it will not follow up on it due to the cancellation of bilateral calls for proposals by both the U.S. and Canadian authorities.
Promotion, Dissemination and Exploitation
26. The status of the Agreements should be reinforced in the EU, in order to achieve the same brand as ATLANTIS in the US. The Commission should encourage the publicity of activities under the Agreements, for instance by supporting the invitations of project coordinators to high status stakeholders to participate in their activities.
The EU-Canada programme would benefit from a higher profile, in order to increase the reach of the programme and the potential for generating wider impacts. To succeed in this, more innovative actions need to be implemented, and the introduction of a title like ATLANTIS to help in publicity and marketing is recommended, e.g. MAPLE (Mobility for transatlantic Promotion of Learning Exchanges) or CHAMPLAIN (Collaboration for Higher transatlantic Mobility Promotion, Learning And International Networks). It is also recommended that the OCEANS network is canvassed for views on a possible name, perhaps through a small competition.
While the Commission recognises the need for enhanced promotion of the Agreements in the EU, it will not follow up on this recommendation due to the cancellation of bilateral calls for proposals by both the U.S. and Canadian authorities.
27. A draft dissemination plan should be included in the interim report of the project coordinators. In the case of POM projects a communication plan should be drafted at the start of the project, making use of press releases. Targets for dissemination should include government officials and other projects. A requirement to develop a dissemination/visibility plan could already be part of the application.
Together with the EACEA, the Commission will look into how best to address the issues of communication, dissemination and visibility of/by projects and their results/outputs to relevant stakeholders.
28. Stakeholders at intermediate and higher levels (national organisations of university/VET institutions of the countries/states involved, international organisations, ministries and other relevant stakeholders) should be more closely involved in POMs, at least being kept informed of the start, progress and results of these projects.
Besides mandatory sustainability and dissemination strategies, the Commission already encouraged POM promoters to include relevant stakeholders as project partners and invited them to disseminate systematically results to these organisations and bodies.
29. The OCEANS network has the potential to make a significant contribution to promotion and dissemination of the Agreements. They should be actively encouraged and supported in this role by the EC and the US and Canadian federal authorities.
The Commission takes note of this recommendation.
30. The Project Directors' Conference is the only major opportunity for the community of participants to come together and exchange views. It is important that it is maintained.
The Commission fully agrees with this recommendation . However, as no new bilateral projects will be selected in the foreseeable future, the need for organising such a conference has diminished..
31. Stimulate exploitation between educational institutions by offering an on-line tool for Q&A on multi-institutional cooperation. Such a tool should both enable interested parties or ‘new’ applicants to learn from the lessons of predecessors in applying and implementing projects and to disseminate results/outputs.
See comments to recommendations (2) and (13) .
Conclusions
The Commission shares the overall assessment of the evaluators that the Agreements are highly relevant and fulfil their broader and more specific goals, especially so in the context of the EU developing its wider role in the world, as well as the objectives of Articles 165 and 166 TFEU.
The Agreements do not overlap nor conflict in significant ways with a wide range of relevant EU initiatives and offer substantial and tangible benefits for students, faculty and institutions. The interest from stakeholders in the Agreements is increasing steadily, with the numbers of applications substantially going up in both 2009 and 2010. Most of the activities would not have occurred without funding from both the EU and the US or Canada.
As the Agreements are comparatively cost-efficient due to their bilateral funding aspect which has a significant added value at a strategic level, they stand as a model to follow for other industrialised and high-income countries.
In view of the above, the Commission deplores the sudden end of the main cooperation activities implemented under both Agreements.
The evaluators have made a number of suggestions to improve certain aspects of the programmes but found no major shortcomings. The Commission has already taken action with regard to some of the suggestions and will take others of them into consideration in the manner indicated above.
Council decision no. 2006/910/EC (OJ L 346 of 9 December 2006, p. 33 ) .
Council decision no. 2006/964/EC (OJ L 397 of 30 December 2006, p. 14 ) .
Youth = EU-Canada Agreement only.
www.oceans-network.eu.
EU-US Agreement: http://ec.europa.eu/education/eu-usa/doc1901_en.htm.
EU-Canada Agreement: http://ec.europa.eu/education/eu-canada/doc1688_en.htm.
Due to the late adoption of the Agreement, no EU-Canada call for proposals was published in 2006.
Except for 2006 when the EU co-funding amounted to EUR 260 000.
Available for free at: http://www.emqa.eu.
http://ec.europa.eu/dgs/education_culture/eve/index_en.htm.
http://ec.europa.eu/education/external-relation-programmes/doc1949_en.htm.
http://www.tdp-project.de.