Explanatory Memorandum to COM(2016)114 - Implementation of the European Statistics Code of Practice and coordination within the European Statistical System - Main contents
Please note
This page contains a limited version of this dossier in the EU Monitor.
| dossier | COM(2016)114 - Implementation of the European Statistics Code of Practice and coordination within the European Statistical System. |
|---|---|
| source | COM(2016)114  |
| date | 07-03-2016 |
Contents
- Brussels, 7.3.2016
- Summary of strengths and recommendations
- 2.1 Governance and legal aspects; coordination
- 2.1.1 Governance and legal aspects, including the legal and procedural aspects of confidentiality
- 2.1.2 Coordination within the statistical system
- 2.2 Adequacy of resources and cost-effectiveness
- 2.3 Quality and methodology
- 2.3.1 Quality commitment
- 2.3.2 Methodology and protection of confidentiality
- 2.3.3 Output quality and user interaction
- 2.4 Burden reduction and administrative data
- 2.5 Impartiality and dissemination
- 2.6 Divergent views
COM(2016) 114 final
REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL
on the implementation of the European Statistics Code of Practice and coordination within the European Statistical System
1. Introduction
The European Statistics Code of Practice (CoP) 1 was endorsed by the Statistical Programme Committee in 2005, and amended in 2011 by the European Statistical System Committee (ESSC) following the adoption of Regulation (EC) No 223/2009 of the European Parliament and of the Council on European statistics 2 .
A self-assessment against the Principles of the Code was launched in 2005. This was followed in 2006-2008 by peer reviews across the Member States, European Free Trade Association (EFTA) countries and Eurostat. These reviews assessed the institutional environment and dissemination practices of statistical institutes (Principles 1 to 6 and 15) 3 . The 2008 Commission report to the European Parliament and the Council 4 concluded that ‘full compliance with the Code remains a challenge for basically all statistical institutes and Eurostat’ despite there being ‘overall high compliance levels complemented by dynamic progress with regard to improvements’. The report envisaged another round of peer reviews ‘…within the next five years subject to advice by the European Statistical Governance Advisory Board in particular on the scope and considering costs and benefits’. Special Report No 12 // 2012 5 by the European Court of Auditors on improvements to the process for producing reliable and credible European statistics also recommended a new round of peer reviews.
Consequently, a new round of peer reviews was launched in the 28 Member States, the four EFTA countries and Eurostat at the end of 2013, within the following five years as envisaged in the above Commission report to the European Parliament and the Council. Their aim was primarily to improve the efficiency and credibility of the European Statistical System (ESS), to increase its capacity to produce European statistics, to reassure stakeholders of both the quality of European statistics and the trustworthiness of the ESS, and to give producers of statistics an inside view on the progress made to date.
This round of peer reviews differed in many respects from the previous one. Firstly, all 15 Principles of the Code and the coordinating role of the National Statistical Institutes (NSIs) within their statistical systems were subject to an audit-like review. Secondly, the reviews focused on issues where further progress was needed. Thirdly, in addition to the NSIs, a number of other national authorities (ONAs) responsible for producing European statistics were assessed, although not to the same extent as the NSIs themselves. Fourthly, evaluations were conducted by independent peer reviewers, with Eurostat being reviewed by the European Statistical Governance Advisory Board. The reports 6 present the peer reviewers’ views and findings and make recommendations for action. The countries concerned and Eurostat responded to these recommendations, designing improvement actions to be implemented accordingly.
2. Main findings and recommendations by issue
The peer review reports highlight a number of strengths. In particular, the following aspects emerged as strengths:
The reports emphasise the strong legal basis that supports the widely recognised and unchallenged professional independence, impartiality and objectivity of statistical authorities. Coordination with ONAs is also seen as a strength for a number of NSIs, as is their clear mandate for data collection.
Highly trained, motivated and dedicated staff contributes to the high level of credibility of and trust in the NSIs. Nonetheless, further reductions in human and financial resources, combined with additional cost-efficiency measures, risk jeopardising their ability to meet future challenges.
Strengths were observed in systems, tools, methods and interaction with users. Nevertheless, the reports indicate a need to improve administrative and operational structures, for example, by introducing quality audits and dedicated quality teams or managers.
NSIs’ strengths lie also in the clear legal bases for obtaining access to administrative data and for increasing cooperation with owners of administrative data. The reports still, however, recommend involving the NSIs more in the design and development of methods and tools for administrative databases.
Significant progress has been made in limiting the number of and in regulating privileged access (under embargo) to statistical pre-releases. The availability of modern tools, visualisation techniques and web-based and social-media data are additional strengths in the area of dissemination.
In addition, the reports mention initiatives such as internal quality reviews, mandatory technical specifications for data submissions, integrated customer relationship management systems, response burden barometers and networks for exchanging experiences. The active participation of members of the ESS on the international statistical scene is considered an asset for the whole system.
The peer reviews led to 707 recommendations, an average of 21 per country/Eurostat. Table 1 below presents an overview of the recommendations, which require action to either achieve compliance or, where compliance has already been achieved, to improve implementation.
Although it is not appropriate to draw comparisons with the previous peer review round due to the differences in its scope and in the methodology used, it should be emphasised that only 9 % (65) of the recommendations concern non-compliance with the Code. Most relate to impartiality and dissemination (22) and professional independence (20). These figures should, however, be interpreted with caution as the recommendations cover a wide array of issues of varying importance, ranging from amending statistical law to placing a logo on a website or in publications.
| Table 1: Peer Reviewers' recommendations by issue | |||
| Issue | Total recommendations | To improve implementation of the CoP | To achieve compliance with the CoP |
| Total | 707 | 642 | 65 |
| 1 Governance and legal aspects; coordination | 216 | 194 | 22 |
| 1.1 Governance and legal aspects | 95 | 73 | 22 |
| 1.1.1 Professional independence | 88 | 68 | 20 |
| 1.1.2 Mandate for data collection | 4 | 3 | 1 |
| 1.1.3 Legal aspects of confidentiality | 3 | 2 | 1 |
| 1.2 Coordination | 121 | 121 | 0 |
| 2 Adequacy of resources and cost-effectiveness | 103 | 99 | 4 |
| 2.1 Resources | 42 | 38 | 4 |
| 2.2 Training | 14 | 14 | 0 |
| 2.3 Cost-effectiveness | 47 | 47 | 0 |
| 3 Quality and methodology | 168 | 158 | 10 |
| 3.1 Quality commitment | 64 | 59 | 5 |
| 3.2 Methodology and protection of confidentiality | 46 | 43 | 3 |
| 3.3 Output quality and user interaction | 58 | 56 | 2 |
| 4 Burden reduction and administrative data | 68 | 61 | 7 |
| 4.1 Burden reduction | 22 | 22 | 0 |
| 4.2 Administrative data | 46 | 39 | 7 |
| 5 Impartiality and dissemination | 152 | 130 | 22 |
| 5.1 Impartiality | 36 | 22 | 14 |
| 5.2 Dissemination: accessibility and clarity | 85 | 78 | 7 |
| 5.3 Dissemination of microdata | 31 | 30 | 1 |
The presentation of the findings below is structured according to groups of issues. These were identified by the peer reviewers based on the structure of the Code, although the groupings are not identical to those used in the Code.
The reports also include the views of the NSIs and Eurostat, in cases where their views diverge from those of the peer reviewers. The NSIs and Eurostat designed improvement actions in response to the recommendations 7 , which are published separately.
This section discusses the independence of statistical authorities, and covers the following specific issues: the nomination and dismissal of heads of statistical authorities; issues related to work programmes; the mandate for data collection; the legal aspects of confidentiality; and the coordinating role of NSIs.
2.1.1 Governance and legal aspects, including the legal and procedural aspects of confidentiality
Two main issues relating to governance emerged from the peer review reports, namely the need to revise statistical laws in order to create greater professional independence in relation to the nomination and dismissal of the heads of statistical authorities, and the need to strengthen stakeholder consultation. Legislative or institutional changes may be required in order to be able to implement many of the recommendations in this area. Implementation of these recommendations therefore falls within the remit of the relevant national authorities rather than that of the NSIs.
In their reports, the peer reviewers focused in particular on the independence of the heads of statistical authorities. The recent amendment to Regulation (EC) No 223/2009 8 further emphasised the need for independence. It requires Member States to take measures to bring their national practices and procedures into line with the new European legal framework. Thus, in order for the ESS to comply fully with Principle 1, there is need to take further action. In particular, the procedures need to be amended such that: i) the appointment of heads of statistical authorities is based on professional competence and open selection procedures; ii) there are clear criteria set out for their dismissal, and these criteria are applied; iii) heads of statistical authorities have sole responsibility for deciding on statistical methods, procedures and the content and timing of statistical releases. The results of the latest round of peer reviews contrast with those from 2006-2008, where professional independence was seen to be fully met. This is due to: i) differences in the methodology used for conducting the peer reviews; ii) the inclusion of ONAs in the bodies reviewed; and iii) changes making the requirements related to the independence of the heads of statistical authorities more stringent.
Statistical authorities are, as a rule, part of the national administration. In order to achieve full independence as defined in the Code, it is therefore essential to ensure that any ministerial or other supervision is of a purely administrative nature and has no implications for policy. The alternative is to make statistical institutes fully independent.
ESS members were generally found to be complying with the Code insofar as concerns the requirement to establish and follow multiannual and annual work programmes, with only one case of non-compliance identified.
Input from users of statistics is important for ensuring that statistics are produced efficiently and that they meet user needs. Advisory or consultative committees should be set up, or their roles clarified where they already exist, and given adequate resources. This will help NSIs to make informed decisions about ongoing and future work.
Principles 2 and 5, which relate, respectively, to data collection and the legal aspects of confidentiality, are being complied with. There is, however, scope for progress in adding further safeguards to ensure that confidential statistical data are not used for administrative purposes.
Coordination within the statistical system does not fall within the scope of the Code. It was, however, included in the exercise, in order to gauge the level of coordination that NSIs provide within their national statistical systems, and thus to be able to further strengthen the ESS.
Most of the recommendations in this area relate to steps that could be taken to strengthen the coordinating role of the NSIs in general. For effective coordination to be possible, the concept of official statistics needs to be clarified and the producers of official statistics need to be designated, where this has not yet been done. There should be a clear legal basis for coordination, including for the related institutional arrangements. The necessary human and financial resources must also be provided and written agreements concluded with ONAs, in order to support the implementation of this legislation in practice. Coordination should be organised on the basis of the common objectives and annual statistical programmes that apply to all bodies within the National Statistical System (NSS). Reports should be published on the implementation of these national statistical programmes.
Moreover, further work is needed to ensure the quality of European statistics produced by ONAs. Under amended Regulation (EC) No 223/2009, the NSIs are responsible for coordination of quality monitoring at national level. In addition, they should provide NSS-wide guidelines, standards and business process models and be the Commission’s sole point of contact on European statistics.
Adequacy of human and financial resources is a major issue across the ESS and was already highlighted as such in the 2008 Commission Report. This issue presents a serious risk for the future of the ESS. In spite of constant resource shortage, the ESS has managed, through reforms and efficiency measures, to cope with the everincreasing needs for more statistical data. The peer reviewers point out, however, that many NSIs are operating under pressure, often to the detriment of methodological improvements and national statistical demands (see also 2.3.1 below). In order to ensure that reliable European statistics can continue to be produced, reviewers generally urge that adequate human and financial resources be secured for the NSIs, as it would be difficult for NSIs to meet further European statistical requirements without compromising on quality.
Faced with these difficulties, many NSIs struggle to attract and retain highly skilled staff. Whereas the situation may vary from country to country, it is generally recommended that the use of human resources be further improved by means of appropriate training plans, transfer of skills, internal mobility, knowledge sharing, increased training in ONAs and staff exchanges within the NSSs.
Cost-effectiveness is closely linked to the effective use of resources (Principle 10). Although no cases of non-compliance were reported, overall cost-effectiveness could be improved in a number of ways, including: i) adopting a statistical business process model such as the Generic Statistical Business Process Model; ii) using standard data processing tools; iii) making wider use of administrative data; and iv) potentially transferring responsibilities from ONAs, especially small ONAs, to the NSI.
There are a number of ESS initiatives currently under way that aim to better balance the increasing demands for European statistics with the available resources. These initiatives should thus help to improve compliance with Principles 3 and 10. Their specific objectives include: simplifying European legislation, prioritising statistical needs in the European Statistical Programme, developing methodologies so as to provide common tools and instruments and ensuring that European statistical requirements are reviewed regularly through Eurostat’s enhanced coordinating role.
European statistics must be underpinned by high quality, sound methodology and appropriate statistical procedures (Principles 4, 7 and 8).
Overall, the ESS should invest more in comprehensive quality management systems, ranging from quality guidelines and tools and the systematic documentation of statistical processes to standard quality assurance procedures and regular quality audits. The peer review reports also raise the need to set up dedicated quality and methodology units with sufficient numbers of suitably qualified staff. The reports highlight the risk the strained human resource situation represents for the future quality and reliability of European statistics, in particular in view of its potential consequences for systematic work on quality. In view of this, adequate resources should be allocated to work on quality within the global allocation (see also point 2.2).
The ESS largely complies with the Code’s methodology-related principles, with only one recommendation (out of 32) being issued on compliance. The reports, nonetheless, recommend improving the methodology for the development, updating and application of standard guidelines on issues such as questionnaire testing, handling sampling/non-sampling errors, editing, imputation and validation rules, seasonal adjustment and estimation and revision policy. There should also be closer cooperation with the scientific community on methodological work.
The peer review reports highlighted a number of issues relating to the protection of confidentiality, mainly where improvements could be made to protection techniques, but overall compliance is satisfactory in this area. The main issues to be addressed are establishing the necessary procedures to monitor the use of microdata and adding further safeguards to ensure that confidential statistical data are not used for administrative purposes.
The ESS complies with the Code in the areas of output quality and user interaction. The only two recommendations (out of 58) where compliance was considered not yet to have been achieved relate to the policy on revisions and calendars for statistical releases.
The peer reviewers call in particular for monitoring and improving timeliness and punctuality of statistical releases as well as for user-friendly release calendars which should be regularly updated.
The recommendations also call for greater commitment to meeting user needs. Steps should be taken to raise awareness among users and build more open relationships with the news media.
The issues identified by the peer reviewers mostly relate to the persisting obstacles to access to and use of administrative data, and to ensuring that NSIs are informed and involved at an early stage when administrative data sources are to be set up, modified or discontinued (indicators 2.2, 8.1, 8.7 to 8.9). Although most NSIs have statutory access to or other appropriate arrangements allowing them to access administrative data, in practice, they may only be granted access reluctantly.
Problems related to accessing administrative records have a significant impact on the response burden (Principle 9). Over 15 % of the recommendations related to accessing administrative records are about compliance with the Code. Any strategic approach to addressing this issue should therefore cover the legal or administrative arrangements, including the written agreements that are concluded with the owners of administrative data to allow NSIs to have full access to all the administrative data they need for producing European statistics, including tax records. Overall, a general change of attitude should be advocated, such that compiling official statistics comes to be considered as one of the purposes of administrative registers. Statistical authorities should be able to influence decisions relating to various aspects of administrative data, in particular: the concepts used, their coverage, the procedures for collecting them, the criteria used to determine their suitability for statistical purposes and their design.
The peer reviewers also recommend a number of other measures designed to ease the response burden without compromising user needs. These measures include data sharing, identifying the statistical potential of new data sources, in particular of big data, and measuring, monitoring and reporting on the response burden on a regular basis.
Despite data being made available on the web and the research community being granted access to microdata in accordance with the Code (Principle 15 and indicator 5.6), the fact of pre-release access being given to privileged users, for example the government ministers concerned (indicator 6.7), continues to be an issue in some countries. The reports recommend that pre-release access rights be strictly regulated, and only granted as an exception. In such cases, the situation should be closely monitored and information on the rights granted made publicly available. To this end, annual release calendars, covering all statistical releases and major revisions that have fixed dates, should be published well in advance. Changes and delays should be explained.
The peer reviewers recommend improving dissemination channels and techniques in particular. They recommend publishing more in-depth and custom-designed analyses, providing modern visualising and dissemination tools, upgrading or redesigning webpages, and improving the coverage and timeliness of data. National websites should have more content in English in order to make them more accessible to other ESS users.
NSS need to do more to ensure that the dissemination of statistics is impartial. The most pressing issue is to ensure that all official and European statistics are disseminated via a single web portal or by creating links on the NSI website, in order to allow users to obtain a full picture of official statistics.
Lastly, the network of safe centres and remote access facilities should be further extended and facilities improved to give better access to microdata for research purposes.
The reviews followed an audit-like methodology, and the reports therefore also contain the statistical authorities’ views where these differ from the peer reviewers’ findings and recommendations. As shown in Table 2 below, there are 39 instances where the statistical authorities’ own views diverged from those of the peer reviewers. As a rule, where there was a difference of opinion, no improvement actions have been designed. Compared with the total number of recommendations (707), the number of issues on which views diverge remains limited. This means that the peer reviews were successful in identifying issues for improvement.
By far the highest number of divergences of opinion (18 out of the 39 divergent views) was found in the area of governance and legal aspects, in particular professional independence. Unlike the peer reviewers, the statistical authorities in these cases considered existing legislation or other arrangements to be sufficient to guarantee professional independence. This was particularly the case with respect to the recruitment and dismissal of heads of statistical authorities, where procedures were said to be in line with the general policies of the national authorities to which the statistical authorities belong.
The remaining divergent views were spread relatively evenly over the other categories of issues, with up to four divergent views being recorded for each. In each of the three areas, the protection of confidentiality, the dissemination of microdata and coordination, the authorities’ views differed from those of the reviewers in three cases. In the case of insufficient protection of confidentiality and dissemination of microdata, the peer reviewers’ recommendations were contested, with statistical authorities citing the risks that would be posed to burden reduction, cost-efficiency or methodology were the recommendations to be accepted. The diverging views recorded on the issue of coordination were country-specific.
| Table 2: Divergent views on peer reviewers’ recommendations | ||
| Issue | Recommendations | Divergent views |
| Total | 707 | 39 |
| 1 Governance and legal aspects; coordination | 216 | 21 |
| 1.1 Governance and legal aspects | 95 | 18 |
| 1.1.1 Professional independence | 88 | 16 |
| 1.1.2 Mandate for data collection | 4 | 1 |
| 1.1.3 Legal aspects of confidentiality | 3 | 1 |
| 1.2 Coordination | 121 | 3 |
| 2 Adequacy of resources and cost-effectiveness | 103 | 3 |
| 2.1 Resources | 42 | 1 |
| 2.2 Training | 14 | 0 |
| 2.3 Cost-effectiveness | 47 | 2 |
| 3 Quality and methodology | 168 | 7 |
| 3.1 Quality commitment | 64 | 2 |
| 3.2 Methodology and protection of confidentiality | 46 | 4 |
| 3.3 Output quality and user interaction | 58 | 1 |
| 4 Burden reduction and administrative data | 68 | 3 |
| 4.1 Burden reduction | 22 | 2 |
| 4.2 Administrative data | 46 | 1 |
| 5 Impartiality and dissemination | 152 | 5 |
| 5.1 Impartiality | 36 | 1 |
| 5.2 Dissemination: accessibility and clarity | 85 | 1 |
| 5.3 Dissemination of microdata | 31 | 3 |
3. Improvement actions
The NSIs and Eurostat have developed improvement action plans in response to the peer reviewers’ recommendations. These follow the SMART approach. 9 The exercise led to a total of 929 improvement actions.
The implementation of improvement actions will be monitored annually. NSIs and Eurostat report on their progress, and the information they provide then feeds into both Eurostat’s annual progress reports to the ESSC, and the European Statistical Governance Advisory Board’s annual reports to the European Parliament and the Council. In order to maintain momentum, all improvement actions should be implemented by the end of 2019, after which date Eurostat will prepare a final report for the ESSC.
4. Conclusions and next steps
The ultimate objective of the assessments conducted in this round of peer reviews is to strengthen statistical systems at both national and European level, thereby increasing the trustworthiness and credibility of European statistics and equipping the ESS to tackle future challenges. The adoption of the Commission Decision 10 on the enhanced role of Eurostat in coordinating the production of European statistics, the adoption in May 2015 of the Regulation amending Regulation (EC) No 223/2009, which significantly strengthens the independence of NSIs and their role in coordination, and the developments relating to the ESS Vision 2020 11 also underline the importance of this peer review round as a crucial tool in modernising the ESS. Although the exercise has value in itself by making ESS members scrutinise their policies, practices and procedures, its full impact will not be felt until the implementation of recommendations has progressed further.
In general, the ESS has shown a high level of compliance with the Code. Statistical authorities operate on sound legal bases, their credibility and independence are on the whole widely recognised and they enjoy a high level of trust. While there are significant concerns related to resources, the highly professional, well-educated and motivated staff is a significant strength for the ESS. The methodologies used and the quality of the statistics have improved, thanks to further standardisation and harmonisation of processes. Nonetheless, a number of challenges remain, in particular with respect to improving and fine-tuning the implementation of the Code throughout the ESS.
While the mandate of statistical authorities and their activities are determined by their legal basis, some aspects and practices should be strengthened or revised. This relates in particular to the development of clear, transparent and objective procedures for recruiting and dismissing heads of statistical authorities. Although several statistical authorities disagreed with the peer reviewers on the grounds that they follow the general administrative procedures, this issue would merit further reflection.
Access to and effective use of administrative data is a critical issue in achieving cost-efficiencies and reducing the response burden. Although legislative compliance has been achieved in this area, as the legal bases for NSIs generally allow access, some practical obstacles to actually obtaining this access still remain. A step forward noted by peer reviewers in a number of countries was statistical authorities being involved in deciding on any planned changes to or the discontinuation of registers, and on the design of any new registers.
The coordination of official statistics presents a varied picture, partly due to the NSSs being centralised to varying degrees, and to the size of the respective countries. There are a number of statistical authorities that have a strong legal basis and an effective and recognised mandate to coordinate the work of ONAs. In other cases, however, coordination is informal, based on ad hoc arrangements and personal contacts. This may be seen as acceptable, in particular in smaller countries, but this type of arrangement does not allow NSIs to be invested with the power needed to ensure the quality of European statistics. In order for the NSIs to coordinate their national systems effectively, the coordinating role should therefore be backed by the necessary legal provisions and institutional arrangements.
Methodology and quality have benefited from increased cooperation at ESS level, with both harmonised and standardised tools and procedures and more extensive sharing of knowledge and experience being introduced.
Resources are an area of concern. All statistical authorities have optimised or are optimising resource use by means of a range of measures, including determining priorities, making use of administrative data and introducing automated data collection. Nonetheless, new and stricter policy requirements mean that the ESS is thought to be approaching a critical threshold in terms of balancing increasing statistical needs and resource limitations. The peer reviewers almost unanimously emphasise that securing adequate resources while also maintaining flexibility in terms of how ESS members use these resources is of the utmost importance for safeguarding the quality of European statistics.
In conclusion, this round of peer reviews has demonstrated both the dynamic development that has taken place in the ESS in recent years and the value of European cooperation for its work. In response to the recommendations, the NSIs and Eurostat have prepared improvement action plans setting out practical measures to improve their performance. Implementation will be subject to close annual monitoring as of spring 2016 and should be completed by the end of 2019. Future measures will be decided in the light of lessons learnt and shared experiences.
(1) ec.europa.eu/eurostat/documents/3859598/5921861 .
(2) OJ L 87, 31.3.2009, p. 164 .
(3) All references to Principles and Indicators relate to the European Statistics Code of Practice.
(4) COM(2008) 621 final, 2008 Report from the Commission to the European Parliament and the Council on implementation of the Code of Practice (7.10.2008).
(5) www.eca.europa.eu/Lists/ECADocuments/SR12_12">www.eca.europa.eu/Lists/ECADocuments/SR12_12 .
(6) ec.europa.eu/eurostat/web/quality/peer-reviews .
(7) ec.europa.eu/eurostat/web/quality/peer-reviews .
(8) OJ L 123, 19.5.2015, p. 90.
(9) Improvement actions should be specific, measurable, achievable, realistic and time-bound.
(10) Commission Decision (2012/504/EU) of 17 September 2012 on Eurostat (OJ L 251, 18.9.2012, p. 49)
(11) ESS Vision 2020