Annexes to COM(2008)334 - Final evaluation of the eTEN Programme - Main contents
Please note
This page contains a limited version of this dossier in the EU Monitor.
| dossier | COM(2008)334 - Final evaluation of the eTEN Programme. |
|---|---|
| document | COM(2008)334  |
| date | June 3, 2008 |
The Commission welcomes the evaluation report and its findings confirming both the effectiveness of the programme management and the utility of the programme.
It agrees that the potential for synergy between the eTEN programme and the European Cohesion Fund might have been better exploited.
It also recognises some weaknesses identified by the professional evaluators, notably on the time from closure of call to contract award and long project time-scales. It notes that these weaknesses have however had a minor effect on the overall programme implementation: The Commission has already initiated new services which will reduce such delays in the future.
It welcome the positive assessment of the programme design; the efficiency of the programme management; and the programme's potential to deliver sustainable results, generate positive impacts and add real value by mobilising all of the functions necessary to successful ICT deployment and catalysing structural changes. The results provide also evidence that the Policy Support Programme of the Competitiveness and Innovation Programme (CIP) can be expected to build on the results of eTEN.
It considers that implementing such a unique set of funding mechanisms has been a particular challenge for all actors involved, and welcomes that the evaluators have recognised the efforts which resulted in major improvements in the late stages of the programme.
5. CONCLUSIONS OF THE COMMISSION
The Commission considers that this is a very positive report confirming both the effectiveness of the programme management and the utility of the programme. Some weaknesses have been identified but these have had a minor effect on the overall programme implementation.
Immediate impacts are the involvement of stakeholders from New Member States, SMEs and public bodies favouring the further deployment and uptake of project outputs at a pan-European level and the competitive health of markets for these and related services. Based on the results so far the programme may thus reasonably be expected to produce strong overall impacts by stimulating new areas of activity and demonstrating the ways in which ICTs may contribute to effective policy relating to innovation and the Lisbon Agenda and i2010 objectives. The results provide evidence that the follow on programme, the Policy Support Programme of the CIP, can be expected to build effectively on the results of eTEN.
Work will continue into 2009 by both the evaluators and the Commission to determine the impact of projects from later years of the eTEN Programme. The Commission will report on this work in the midterm evaluation of the Policy Support Programme of the CIP.
Annex 1: Key Findings from the Evaluator's Report
RAND Europe has undertaken a final evaluation of the eTEN programme. This programme nominally finished at the end of 2006 although some project activities contracted late in its life continue. Key findings of the evaluation are as follows:-
Overall success of the Programme
eTEN was ultimately a well run and, in its later stages, successful programme, which contributed to the attainment of its over-arching objectives. This finding is, however, qualified in two ways. Success was not achieved without difficulties, particularly in the early years of the programme. Additionally the extent of its success is not quantifiable – in part because its impacts are not yet fully visible and in part because its overarching objectives were expressed in qualitative terms without specific measurable success criteria.
Programme Implementation Mechanisms
Funding mechanisms other than grant aid were not utilised: this is unsurprising given the modest size of the projects, and we do not consider that this significantly affected the programme’s performance. Linkage to the European Regional Development Funds and the European Social Fund was weak, despite the apparent scope for benefits.
The Market Validation (MV) project mechanism was initially over-utilised (in part because of its more favourable financial arrangements for participants); this was remedied in the later stages of the programme. Many MV projects commissioned before the reorientation did not result in any visible subsequent deployment; this situation also improved in the later stages of the programme.
The Initial Deployment (ID) project mechanism came on stream late, but worked well, particularly following the much-delayed implementation of the 30% cap on overall support as a proportion of total cost. Some promising deployments are now in progress and prospect, including some originating from earlier MV projects.
The funding contracts allowed for insufficient flexibility. Project reviews were consequently limited in their ability to either direct or sanction necessary major changes of direction during project implementation. In interviews, some project managers reported an impression that they had no realistic means of modifying this situation even when the problem had been recognised.
Efficiency of Programme Management
From 2003 no major management issues arose and the standards of management processes; fairness, openness and transparency, and efficiency of day-to-day management were excellent.
The operational procedures for preparing Work Programmes and Calls, their accompanying documentation and the receipt and evaluation of proposals improved continuously throughout the programme. Significant initial problems (particularly with proposal evaluation) were overcome and these mechanisms were subsequently satisfactory and, by the end of the programme, became excellent.
Objective formulation and its reflection in work programme creation were sound and appropriate: we found no reason to criticise the evolution of objectives and indeed consider this responsiveness to change in the programme’s external market (and policy) environment a very positive feature.
The process leading to contracting projects (a sequence consisting of selection, credential checking, and negotiation of the technical annexes to contracts) took too long. This had been noted as a problem by the Intermediate Evaluation, and continued thereafter. In addition, project time-scales were long relative to private-sector counterparts and the dynamics of fast-moving sectors. This, together with the delays in contracting projects, frequently resulted in a very long ‘time to market’, which is likely to have worked against successful deployments and reduced their potential impacts.
Effectiveness, Achievements and Impacts
The programme made considerable progress in involving stakeholders from New Member States, SMEs and public bodies. Their participation strongly favours the further deployment and uptake of project outputs at a pan-European level and the competitive health of markets for these and related services.
Activities in the latter stage of the programme promise good impacts through the formation of sector-based value chain communities (especially in the public sector). The emphasis placed in the latter stages of activity on inclusion in projects of the full value chain including ‘sustaining partners’ and ‘sustaining revenue streams’ was fully validated and represents an important lesson for future programmes.
Fulfilment of Programme Objectives
The later stages of the programme we find to have added real value through actions seeding and/or exemplifying structural change and also through placing actual deployment at the centre of policy development. The programme may thus reasonably be expected to produce strong overall impacts by stimulating new areas of activity and demonstrating the ways in which ICTs may contribute to effective policy relating to innovation and the Lisbon Agenda and i2010 objectives.
Sustainability of Activities
Long-term sustainability of activities is difficult to assess at this stage. However, the involvement of the whole value chain (adopted in the later and more productive stages of the programme), and the specific example provided by some of the projects that have proceeded into wider deployment via these value chains, strongly suggests a positive prognosis.
[1] The Trans-European Network (TEN) initiative is based on articles 154, 155 and 156 of the Treaty establishing the European Community, which set the objective of establishing Trans-European networks in the areas of transport, telecommunications and energy.
[2] COM (2007) 0135 Communication from the Commission – Trans-European networks: Towards an integrated approach {SEC(2007) 374}.
[3] Guidelines for growth and jobs (2005-2008) No 9, 10, 11 and 16.
[4] Regulation (EC) No1655/1999 of the European Parliament and of the Council of 19 July 1999, amending Regulation (EC) No 2236/95 laying down general rules for the granting of Community financial aid in the field of trans-European networks (OJ L 282, 24.11.1995, p. 16).
[5] Article 5 of Regulation No 1655/1999.
[6] Article 4 para 1(a) of Regulation No 1655/1999.
[7] Regulation (EC) No 1159/2005 of the European Parliament and of the Council of 6 July 2005
[8] Decision No 1376/2002/EC of the European Parliament and of the Council of 12 July 2002 amending Decision No 1336/97/EC on a series of Guidelines for trans-European telecommunications networks.(OJ 2002/ L200/1 of 30.07.2002)
[9] COM (2005)354 final, 1 Aug 2005.
[10] RAND Europe, contracted on the basis of a restricted call for tenders issued by DG Information Society in the autumn of 2006.