Annexes to COM(2017)113 - Ex post evaluation report on the Drug Prevention and Information Programme (2007-2013) - Main contents
Please note
This page contains a limited version of this dossier in the EU Monitor.
| dossier | COM(2017)113 - Ex post evaluation report on the Drug Prevention and Information Programme (2007-2013). |
|---|---|
| document | COM(2017)113  |
| date | March 6, 2017 |
More funds could have been committed to the programme, in particular with a view to achieving the objective of reducing drug demand. The very high number of applications for funding under the DPIP (as opposed to the small number of funded projects), and the fact that the most commonly funded organisations were NGOs who largely rely on external financing, confirms the general need for adequate financial resources in the area of drugs policy. Overall, the financial resources available were used in an efficient way, judging by the comparison of inputs/outputs in projects’ budgets. 46
Though some of the expected impacts of DPIP were very ambitious as they derive from the programme objectives, the implemented projects achieved good number of outputs and positive results which suggest that the amount of money spent was reasonable in comparison to the achievements. The impact of the projects is demonstrated in the development of new prevention tools; new treatment methods corresponding to the latest state of scientific knowledge in the drugs field; and awareness-raising approaches targeting specific (vulnerable) groups. Furthermore, the high demand for DPIP grants has allowed the Commission to select those projects which showed the most potential and represented the best value for money. 47
Figure 22 Total allocated, committed and paid DPIP funds and total over/under commitment and underspending per call (action grant above, operating grant below)
Note: No information was available on paid funds and underspending for 2011-2012 action grants
DPIP — Calls for Proposals (2007-2012) 2013 could not be considered within the scope of this evaluation, because the final project reports were not submitted during this evaluation process.
2.5.2Extent to which the money spent was reasonable in view of the impact achieved
The DPIP was expected to:
• improve the prevention of drug use in the EU;
• create better understanding/improved information on drugs and associated harm in the EU;
• improve the implementation of the EU Drugs Strategy and specific actions under the action plans.
It is reasonable to expect the programme to achieve its objectives with the resources available, especially with regard to improving the provision of information on drugs and associated harm in the EU. Nevertheless, the first objective, i.e. improving prevention, was the most challenging to achieve given not only the scale of the problem of drug use versus the size of the programme, but also the fact that reducing drug use is inherently associated with behavioural changes and lifestyle choices which are generally difficult to influence and may require a series of long-term interventions. The second objective was worked on through the dissemination of outputs and results by the Commission and individual projects and networks (more on this see the section 2.4.1 Continuation of project activities and outputs and dissemination). The third objective — better implementation of the EU Drugs Strategy and action plans — was achieved mainly through a series of high-level policy reports covering a host of issues relating to drugs policy in the EU. These policy reports were outsourced to external contractors. 48
The evidence collected as part of this evaluation is still insufficient to firmly conclude that the resources spent on the programme were reasonable in view of the long-term outcomes and impact achieved.
On the other hand, the outputs and results of DPIP action grants and operating grants which form the basis for the long-term outcomes and impact, were positive, in particular in relation to 49 :
• developing new prevention tools, harm-reduction measures and treatment approaches to address new developments and rapid changes as regards drugs, or developing tools (e.g. quality standards for online helplines) supporting the work of organisations dealing with target groups;
• developing new approaches to raise awareness and provide information among specific target groups/vulnerable groups;
• establishing multidisciplinary networks and/or platforms to increase the exchange of information and cooperation between organisations, further disseminate project results/good practices and gain visibility with policy-makers;
• capacity building and training to strengthen the implementation of the EU Drugs Strategy and action plans and relevant actors in prevention of drug use/dependence and associated harm.
DPIP funded 51 mostly transnational projects, representing approximately 200 leading and partner organisations. The analysis of the projects finalised so far shows that at least 63 % of the finalised action grants and operating grants show evidence of having achieved their objectives, although action grants showed slightly less evidence of achieving their objectives (60 %) than operating grants (67 %). 50
2.5.3Appropriateness of the allocation of funds among the different funding tools
According to the annual work programmes, the total budget allocated for implementing the DPIP for 2007-2013 amounted to EUR 22 232 million. The planned budgetary breakdown is presented in Table 1 1 Planned budgetary breakdown for DPIP (2007-2013) . The average value of action grants increased between 2007 and 2012, with most of the funding allocated during the second half of the implementing period. The budget absorption of action grants (payments as a share of commitments) was acceptable, i.e. 88 %. This suggests that funding for action grants has been allocated in an efficient manner. 51 Note that this analysis does not consider most of the grants awarded under action grants for 2011-2012.
Operating grants received more than half of the funds initially allocated. The average value of OGs decreased considerably between 2008 and 2012 (reaching its lowest value in 2011), ranging from EUR 20 499 (2012 operating grant) to EUR 200 000 (2008 operating grant). The lower average value of operating grants can be explained by their shorter duration (maximum 12 months) and the fact that one operating grant funds only one organisation, whereas a partnership of organisations is financed under action grants. Budget absorption of operating grants was acceptable, i.e. 85 %. In the same way as for action grants, operating grant spending appears to have been efficient. 52
During the implementation period, the Commission committed approximately EUR 2.7 million to a total of 28 procurement contracts (or about 17 % of the total committed budget of DPIP). Approximately half of the procurement budget was committed to the European Action on Drugs (EAD) — a multiannual awareness-raising campaign – and to regular meetings of the Civil Society Forum on Drugs. With an initial allocation of over EUR 4 million, procurement has been under-used and this might have had a negative effect on, for example, the dissemination of programme results at EU level. 53
Finally, the funding tools were all implemented through a competitive process, using calls for proposals (for the grants) and calls for tenders (for procurement) which attracted high numbers of applications. Action grants and procurement contracts are more effective and efficient funding tools. 54
2.5.4Commission management and scope for simplification
Based on the online survey results, the requirements for applicants to access DPIP funding were deemed appropriate overall, as presented in Figure 2 3 below.
Figure 23 Application requirements
Source: Survey for the ex post evaluation of the five DG Justice programmes 2007-2013.
Question 36: With regard to the Commission’s management of the five programmes, including the Commission’s monitoring and evaluation of your project/activities, please comment on the following statements (see Figure above):
The information provided in calls for proposals throughout the funding period was straightforward and explained the various elements required to apply for funding. However, the document became lengthy, partly reflecting the increased level of detail required in the application form, the number of priorities presented and the introduction of the PRIAMOS IT system. The information included in the calls for proposals was clear and easy to understand according to the majority of the respondents to the online survey. The 2013 call introduced the Single Guide for Applicants, covering the five DG Justice programmes, ISEC and PROGRESS. The Single Guide made it easier and more efficient for applicants to submit proposals for different projects.
Throughout the programme, the application form asked applicants to describe the project in terms of general project information, implementation, financial management, results, evaluation and dissemination. In addition, applicants were required to complete budget estimation forms, a staff-cost analysis and a partnership declaration. From 2010, applicants were also asked to provide indicators to assess results, evidence of previous programme experience, add more detail on the partners and work streams. This increased the potential quality of the projects, motivating the applicants to develop a more rigorous plan and estimate the cost of each of their activities.
Overall, the Commission’s management of the DPIP improved over time, thanks to:
• the introduction of the single programme management unit,
• the publication of a handbook for Commission officials,
• indirectly, through the dissemination of a project management guide.
Overall, grant beneficiaries’ experience of cooperation with the Commission was positive. The grant application requirements followed a similar process to other Commission centrally-managed programmes and required more detailed information, which improved the quality of the applications and the projects. The changes to reporting also created a more balanced approach between financial justification on the one hand and the evaluation/assessment of a project’s results and potential impact on the other.
In 2007, the application form required the following information: the project’s objectives and relevance; project implementation (impact, methodology, duration, concrete outputs and foreseeable risks and difficulties); follow-up, sustainability and visibility (including dissemination and European added value); information on the applicant and partners; a declaration by the applicant; and an application package and a checklist.
In addition to the application form, applicants were asked to submit annexes comprising, among others, partner and associate declarations, co-financing declaration forms, budget forms, a staff-cost analysis form, timetable, financial identification forms, legal entity forms, a declaration on exclusion criteria and CVs.
According to the information collected in the survey, in order to respond to the call for proposals just under half of the respondents (48 %) had to request at least some assistance. Moreover, more than half of the respondents (61 %) confirmed that they knew of potential applicants who did not respond to the calls for proposals due to the complex requirements set by the Commission. These findings are illustrated in Figure 2 4 .
Figure 24 Commission’s management of the five programmes during the application phase
Source: Survey for the ex post evaluation of the five DG Justice programmes 2007-2013.
Question 36: With regard to the Commission’s management of the five programmes, including the Commission’s monitoring and evaluation of your project/activities, please comment on the following statement:
In spite of the difficulties faced by applicants, the total number of applications received for action grants increased steadily from 39 in 2007 to 117 in 2013. However, the number of applications for operating grants decreased from 16 applicants in 2008 and 19 in 2010, to 7 in 2011 and 12 in 2012.
Information on the calls for proposals, both for action grants and operating grants, was available on the DG Justice website. 55 It included documents for applicants, practical information on how to apply, on reporting fraud and irregularities and the projects selected under each call. The vast majority of survey respondents (78 %) regarded the call for proposals clear and easy to understand. Similarly, the procedure for submitting an application for an action grant or operating grant was perceived as straightforward by a majority of respondents (74 %).
Overall, the reporting arrangements concerning the progress and achievements of the projects/activities were considered to be appropriate by 56 % of respondents to the online survey. Moreover, just under half of the survey respondents considered the Commission’s monitoring arrangements at least partially good and helpful during the implementation of the projects and activities. 56
Nearly half (43 %) of the grant beneficiaries who responded to the online survey considered the Commission’s monitoring arrangements as partially good and helpful during the implementation of the project/activities. Moreover, progress reports for grants lasting 24 months or more were regarded as useful by grant beneficiaries since these would provide both project managers and the Commission with an overview of the results achieved so far and would allow them to introduce adjustments to work streams, when relevant. 57
2.6EUROPEAN ADDED VALUE
EU added value refers to the extent to which the EU nature of the programme brings value to its stakeholders and the extent to which the EU has a comparative advantage over national and international actors working in the area. First, the EU nature of the programme and its geographical coverage is discussed. The EU added value is then analysed in terms of value brought to the EU and to beneficiaries. However, it has not been possible to identify and measure EU added value already in terms of impacts.
The DPIP had a strong transnational dimension. This is borne out by the objectives of the programme and by the kinds of action deemed eligible in the 2007 Decision. Article 4 of the Decision states that the DPIP ‘shall support specific transnational projects of Union interest presented by at least two Member States or by at least one Member State and one other country which may either be an acceding or a candidate country.’ Eligible actions also included national projects.
2.6.1Geographical coverage and involvement of Member States
The DPIP covered lead and partner organisations from a total of 25 EU Member States and Norway. Overall, there was a relatively good geographical coverage of activities across the EU. The nature of the programme brought added value to the EU and to most grant beneficiaries. However, some Member States received a larger number of grants and participated in a larger number of partnerships than others (see Figure 2 5 ). Lead organisations were clustered within three Member States: Germany, the United Kingdom and Italy. Together these Member States led 61 % of all projects.
From the total of 457 applications received through the DPIP, the highest number was submitted by Italian organisations (24%, 109), followed by those based in the United Kingdom (60) and Spain (55). These top three applicant Member States submitted 49 % of all applications for funding from the programme. Against this background it is worth noting that the highest success rates of the submitted applications were found to be those from Austria, Germany and the Netherlands.
The partnership structure of the Member States with the highest number of lead organisations (Germany, the United Kingdom and Italy), is shown in Figure 2 6 . Data show that lead organisations are more likely to partner with organisations from their own Member State than with organisations from other EU Member States. This is especially pronounced for Italy. However, this was not the case for the projects led by the United Kingdom, where the lead organisations were most likely to form partnerships with German organisations. For example, Italian lead organisations formed partnerships with six Italian and three UK organisations, while German lead organisations formed partnerships with four German and three Belgian, three Austrian and three Lithuanian organisations. Lead organisations from the United Kingdom formed partnerships with six German, five French and four Italian organisations.
Figure 25 Total number of organisations involved in DPIP action grant projects, including lead (left) and DPIP action grants partner (right) organisations
Figure 26 Partnership structure for the top three Member States of lead organisations
2.6.2Added value for the EU and for grant beneficiaries
The findings of this evaluation show that the main features that brought EU added value to the programme for grant beneficiaries were:
• mutual learning and exchanges, and increased knowledge through cooperation across the borders;
• the ‘EU brand’ and reputation provided by the DPIP;
• EU funding which made the implementation of the projects possible.
The findings of the survey and stakeholder consultations showed that the transnational dimension of the DPIP has allowed organisations established in different Member States to cooperate together and to develop and implement transnational activities aimed at reducing drug demand. While lead organisations are geographically concentrated in just a few Member States, the partnerships they formed allowed a high number of organisations from other Member States to participate, collaborate and learn with/from the project leaders and to implement similar activities. Overall, transnational partnerships contributed to mutual learning, collaboration and to sustainable support both at national and EU level. This is also shown in the results of the stakeholder consultation which are further explored in this section.
The evaluation concluded that the key specific benefits of transnational partnerships while implementing DPIP projects were: the creation of a network of international partners (90 % of survey respondents), an increased knowledge/expertise in the area (85 % of survey respondents) and an increased knowledge of policy and practice in other Member States (70 % of survey respondents). Hence, transnational partnerships also contributed to achieving the specific objectives of DPIP which refer to supporting transnational actions such as creating multidisciplinary networks, expanding the knowledge base, exchanging information and identifying and disseminating good practices. Furthermore, when implementing transnational activities in the area of drug demand reduction, DPIP projects also promoted transnational learning through partnerships. Follow-up interviews with project beneficiaries showed that transnational partnerships helped disseminate best practices, encourage transnational learning and identify information gaps and common issues in drug demand reduction.
The specific benefits of transnational partnerships (see Figure 2 7 ) also resulted from good working experiences between project partners. The majority of survey respondents said they had good working relationships with their transnational partners (74 %) and were satisfied with the partnerships established by the projects (62 %). These positive relationships enabled and strengthened transnational learning — the majority of respondents (68 %) exchanged experiences and lessons learned with their partners during the project implementation. In this respect, 48 % of the survey respondents agreed that it would have been beneficial to involve more partners from different Member States in the project implementation to multiply this effect.
Figure 27 The extent to which project beneficiaries had a good experience with the transnational partnership
Source: Survey for the ex post evaluation of the five DG Justice programmes 2007-2013.
Other form of value that the programme brought was the “EU brand” and reputation of the DPIP. This meant a higher visibility, interest in the projects and their higher impact on EU and national policy-makers, practitioners and the wider public.
Finally, the evaluation showed that EU funding was essential to implement the DPIP projects and achieve the EU objectives. Therefore, the likelihood of achieving the EU’s objectives — and thus meeting the objectives of the EU Drugs Strategy — without EU funding is intrinsically limited. 58
3CONCLUSIONS
Relevance of the programme 59
• Overall, the priorities of the calls for proposals and the selected initiatives were relevant to the objectives of the programme as defined in the legal base. The general and specific objectives of the DPIP are broadly formulated and overly ambitious in the legal act. However, the annual priorities set by the Commission were not only clearly defined, but were also realistic and attainable and addressed the key policy developments in the policy area. They became an increasingly important tool for the Commission to influence the scope of the funded projects throughout the life of the programme.
• The selected actions are relevant to the programme objectives. The project/work programme objectives of all 51 projects were in line with the specific objectives of the programme. Operating grants were only partially aligned with the specific objectives of the DPIP.
• At project level, the selected initiatives appear to have adequately supported policy developments. Procurement contracts were also particularly relevant for developing policy and legislation in this area.
• Overall, the programme was relevant to the needs of grant beneficiaries. The programme is unique in this area and filled a funding gap at national level. Moreover, the conceptual framework of the programme and its priorities matched the needs of actors working in the area of drug prevention.
Coherence and complementarity
• Complementarity of the DPIP with other EU programmes and interventions was almost fully achieved. Complementarity was reached through mechanisms that the Commission put in place at programme design stage and at the stage of designing calls for proposals.
Effectiveness
• Overall, the DPIP was effective in achieving its general programme objectives, although the impact was somewhat limited by the relatively low budget and number of projects funded. The DPIP contributed to fostering intra-European awareness-raising and information on drugs and associated harm, in particular among young people and drug users. Furthermore, DPIP-funded action led to an improved dialogue on drugs, and facilitated an exchange of best practice among stakeholders.
• Some DPIP projects contributed to EU policy-making/legislative development. Procurement contracts, being specifically aimed at developing and implementing policy and legislation in the area of drug prevention, were more effective in this respect because they triggered policy debates and steered the policy-making process at EU and Member State levels.
• At project level, most initiatives achieved their own objectives, in particular thanks to good working relationships with partners and to a clear intervention logic with regard to the target group, objectives, method and activities to implement. No major obstacles were experienced in the implementation of project activities.
• DPIP-funded actions also developed tools recognised as innovative in a number of areas. These tools included prevention and harm-reduction measures and treatment approaches, research methods or contributions to new research in order to fill knowledge gaps and innovative approaches to provide information and raise awareness among vulnerable groups.
Sustainability and dissemination
• Overall, the dissemination of DPIP results helped increase the impact of the projects on the ground, in particular as a result of the dissemination methods put in place, which helped to reach a wide range of stakeholders (EU and national policy-makers, non-governmental organisations/civil society organisations, social workers, young people and drug experts).
• Project results were disseminated through beneficiaries’ websites and networks as well as through Commission and EMCDDA websites. This further improved the visibility of the programme itself (demonstrated by the increasing number of applications received). In addition, public procurement contracts were used to produce information material and organise dissemination events to publicise the results of the projects. However, mechanisms and strategies for dissemination were not exploited to their fullest potential.
• Most initiatives developed a sustainability and/or dissemination plan. However, the extent to which project beneficiaries were effective in securing the sustainability and transferability of project outputs and activities varied substantially.
Efficiency
• The funding made available for implementing the DPIP may not have been entirely sufficient considering the level of ambition of some of the objectives, the high demand for funding, the high absorption rates of grants and the high numbers of outputs and results achieved. Nevertheless, the funding made available under the DPIP was sufficient for beneficiaries to achieve their own objectives.
• The Commission’s management has improved over time and grant beneficiaries experienced cooperation with the Commission as positive.
• As regards scope of simplification, the level of detail required in the application form has increased from the 2010 call onwards and included the introduction of work streams. This has benefited both the Commission and the applicants. Reporting requirements were considered appropriate by the vast majority of the respondents. The Commission’s monitoring arrangements were at least partially considered as good and helpful during the implementation of the project/activities. The reporting requirements reflect a more balanced approach between financial justification on the one hand and evaluation of actual results and potential impacts of the projects on the other.
EU added value
• Most EU Member States participated in the DPIP either as a lead organisation or a partner.
• The EU added value of the programme lay in the fact that the DPIP enabled organisations based in various Member States to make a difference in the area of drug demand reduction. The partnerships formed within the projects helped promote transnational learning and improve the visibility of the initiatives carried out, and also helped identify information in the area of prevention and fight against drugs. The partnerships also improved cross-border cooperation, contributed to the exchange and dissemination of best practices and information, developed mutual trust among Member States and supported the creation of practical tools and solutions to address global challenges. However, the extent of the improvement and contribution, i.e. the EU added value in terms of impacts could not yet be measured.
Key recommendations
• Better define the priorities: the Commission should invest more time and human resources in the process of setting priorities in order to ensure that the priorities can be adequately achieved within an earmarked budget.
• Realistic assessments of project risks and better risk mitigation strategies: the Commission should better monitor risks throughout the project duration, for example by asking for brief progress reports that identify any potential risks as they arise during the implementation of the project.
• Increase focus on assessment of impacts at all levels and not merely on outputs, as regards monitoring and evaluation. This goes hand in hand with the need to collect, analyse and use objective and independent evidence in order to perform project and programme evaluations.
• Explore ways of enhancing the uptake of project outputs, results and best practices by other organisations, including in other Member States, including more resources for translations, communication and dissemination.
• Sharpen the programme's intervention logic; further to the scope of the programme and its general and specific objectives and priorities, types of action and types of intervention and implementing measures, the Commission will seek to sharpen the intervention logic 60 , and make the relations between the rationale, objectives, inputs, outputs, beneficiaries, expected outcomes and impacts more articulate, precise and concrete in any future continuation of the programme.
(1)
Article 15(3)d, Decision No 1150/2007/EC of 25 September 2007 establishing for the period 2007-2013 the Specific Programme ‘Drug prevention and information’ as part of the General Programme on ‘Fundamental Rights and Justice’.
(2)
The ex-post evaluation report by external evaluator is published here: Main report: http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/dpip_programme_evaluation_final_report.pdf , Annexes: http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/dpip_annex_1,_2_and_3.pdf , http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/dpip_annex_4_quantitative_analysis.pdf
Mid-term evaluation report by external evaluator is published here: http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52011DC0246&qid=1467122450426&from=EN
(3)
Article 8, Decision No 1150/2007/EC of 25 September 2007 establishing for the period 2007-2013 the Specific Programme ‘Drug prevention and information’ as part of the General Programme ‘Fundamental Rights and Justice’
(4)
Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Drug Prevention and Information Programme (DPIP), ICF, 28 July 2015, p. 2
http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/dpip_programme_evaluation_final_report.pdf
(5)
Article 12, Decision No 1150/2007/EC of 25 September 2007 establishing for the period 2007-2013 the Specific Programme ‘Drug prevention and information’ as part of the General Programme ‘Fundamental Rights and Justice’
(6)
Not considered within the scope of this evaluation, because the final reports were not submitted during this evaluation process.
(7)
Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Drug Prevention and Information Programme (DPIP), ICF, 28 July 2015, p. 3
http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/dpip_programme_evaluation_final_report.pdf
(8)
Ibid., p. 2
(9)
http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/dpip_annex_1,_2_and_3.pdf
(10)
Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Drug Prevention and Information Programme (DPIP), ICF, 28 July 2015, p. 9
http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/dpip_programme_evaluation_final_report.pdf
(11)
Ibid.
(12)
Ibid., p. 11
(13)
Needs assessment should include relevant and reliable data and should contain a robust analysis clearly demonstrating the need for the action. The applicant can refer to existing research, studies, previous projects which had already identified the need. The needs assessment must make it clear to what extent the action will meet the need and this shall be quantified. The applicant should be specific and focus on the actual needs that the project will aim to address and not limit the analysis to general statements and information about the problems and needs of the target group in general.
(14)
Ibid., p. 11. Based on an analysis of the 23 responses to the online survey received from DPIP grant beneficiaries; and the write-ups of the five follow-up interviews with coordinators of projects/organisations receiving DPIP 2007-2013 grants
(15)
DECISION No 1150/2007/EC OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 25 September 2007 establishing for the period 2007-2013 the Specific Programme ‘Drug prevention and information’ as part of the General Programme ‘Fundamental Rights and Justice’.
(16)
Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Drug Prevention and Information Programme (DPIP), ICF, 28 July 2015, p. 15-16
http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/dpip_programme_evaluation_final_report.pdf
(17)
Ibid., p. 14
(18)
Ibid.
(19)
Ibid.
(20)
http://www.emcdda.europa.eu/
(21)
European Commission (2011) Projects, studies and research on illicit drugs funded by the European Commission, 2007-2013: http://ec.europa.eu/justice/anti-drugs/files/2007-2010_drug_related_projects_en.pdf .
(22)
European Commission (2014) New Psychoactive Substances. Projects, Studies and Research funded by the European Commission: http://ec.europa.eu/justice/anti-drugs/files/nps_report_2014_en.pdf.
(23)
Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Drug Prevention and Information Programme (DPIP), ICF, 28 July 2015, p. 19
http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/dpip_programme_evaluation_final_report.pdf
(24)
Ibid., p. 22
(25)
Ibid.
(26)
Ibid.
(27)
Ibid., p. 23
(28)
Ibid.
(29)
Ibid.
(30)
Ibid., p. 24
(31)
Ibid., p. 25
(32)
Ibid.
(33)
http://ec.europa.eu/justice/anti-drugs/files/equs_main_report_en.pdf
(34)
http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:52013SC0319&from=EN
(35)
Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Drug Prevention and Information Programme (DPIP), ICF, 28 July 2015, p. 27
http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/dpip_programme_evaluation_final_report.pdf
(36)
Survey question 23b: Have the policy makers responded to the information provided by the project/activities? (Multiple choice question). In: Ibid., p. 27. Based on an analysis of the 23 responses to the online survey received from DPIP grant beneficiaries; and the write-ups of the five follow-up interviews with coordinators of projects/organisations receiving DPIP 2007-2013 grants.
(37)
Ibid., p. 28
(38)
Ibid., p. 31
(39)
Ibid., p. 32
(40)
Ibid., p. 33
(41)
Ibid., p. 34
(42)
Ibid., p. 35
(43)
Ibid., p. 36
(44)
Ibid.
(45)
Ibid., p. 39
(46)
Ibid., p. 37
(47)
Ibid.
(48)
Ibid., p. 44
(49)
Ibid., p. 45
(50)
Ibid.
(51)
Ibid.
(52)
Ibid., pp. 45-46
(53)
Ibid., p. 46
(54)
Ibid.
(55)
http://ec.europa.eu/justice/grants1/programmes-2007-2013/drug/index_en.htm .
(56)
Based on an analysis of the 23 responses to the online survey received from DPIP grant beneficiaries; and the write-ups of the five follow-up interviews with coordinators of projects/organisations receiving DPIP 2007-2013 grants. Ex post evaluation of five programmes implemented under the 2007-2013 financial perspective. Specific programme evaluation: Drug Prevention and Information Programme (DPIP), ICF, 28 July 2015, pp. 48-49 http://ec.europa.eu/justice/grants1/files/expost_evaluations_2007_2013/dpip_programme_evaluation_final_report.pdf
(57)
Ibid., p. 50
(58)
Ibid., p. 59
(59)
Ibid., pp. 60-62
(60)
See for instance Figure 1 1 Intervention logic of the Drug Prevention and Information Programme (2007-2013)