EU's Agreement on Reforming Research Assessment

The signatories of the agreement agree:

I. To base our actions on the following Principles

Principles for overarching conditions

• Comply with ethics and integrity rules and practices, and ensure that ethics and integrity are the highest priority, never compromised by any counter-incentives.

• Safeguard freedom of scientific research.

• Respect the autonomy of research organisations.

• Ensure independence and transparency of the data, infrastructure and criteria necessary for research assessment and for determining research impacts.

Principles for assessment criteria and processes

• Focus research assessment criteria on quality. Openness of research, and results that are verifiable and reproducible where applicable, strongly contribute to quality. Openness corresponds to early knowledge and data sharing, as well as open collaboration including societal engagement where appropriate. Assessment should rely on qualitative judgement for which peer review is central, supported by responsibly used quantitative indicators where appropriate.

• Recognise the contributions that advance knowledge and the (potential) impact of research results.

• Recognise the diversity of research activities and practices, with a diversity of outputs, and reward early sharing and open collaboration.

• Use assessment criteria and processes that respect the variety of scientific disciplines, research types, as well as research career stages, and that acknowledge multi-, inter-, and trans-disciplinary as well as inter-sectoral approaches, when applicable.

• Acknowledge and valorise the diversity in research roles and careers, including roles outside academia.

• Ensure gender equality, equal opportunities and inclusiveness.

 II. Implement the following Commitments

The are two main types of commitments; core commitments (1 – 4) and supporting commitments (5 – 10).

The core commitments enable better recognition of the diverse practices and activities that maximise the quality of research, and move away from inappropriate uses of metrics. The supporting commitments enable the move towards new research assessment criteria, tools and processes, and facilitate mutual learning, communicate progress and ensure that new approaches are evidence informed.

Core commitments

1. Recognise the diversity of contributions to, and careers in, research in accordance with the needs and nature of the research

Changes in assessment practices should recognise the broad diversity of: valuable contributions to science and society; practices that contribute to robust, open, transparent and inclusive research; diverse activities (e.g. teaching, leadership, supervision etc.) and diverse roles (e.g. data stewards, software engineers etc.).

2. Base research assessment primarily on qualitative evaluation for which peer review is central, supported by responsible use of quantitative indicators

Research assessment should rely primarily on qualitative assessment for which peer review is central, supported by responsibly used quantitative indicators where appropriate.

3. Abandon inappropriate uses in research assessment of journal and publication based metrics, in particular inappropriate uses of Journal Impact Factor (JIF) and h-index

Inappropriate uses of journal and publication-based metrics in research assessment should be abandoned. This means moving away from using metrics like the Journal Impact Factor (JIF), Article Influence Score (AIS) and h-index as proxies for quality and impact.

4. Avoid the use of rankings of research organisations in research assessment

Recognising that the international rankings most often referred to by research organisations are currently not ‘fair and responsible’ , the criteria these rankings use should not trickle down to the evaluation of individual researchers, research teams and research units. Research organisations should regain the autonomy to shape assessment practices, rather than having to abide by criteria and methodologies set by external commercial companies.

Supporting commitments

 5. Commit resources to reforming research assessment as is needed to achieve organisational changes. 

6. Review and develop research assessment criteria, tools and processes.

7. Raise awareness of research assessment reform and provide transparent communication, guidance, and training on assessment criteria and processes as well as their use.

8. Exchange practices and experiences to enable mutual learning within and beyond the Coalition.

9. Communicate progress made on adherence to the Principles and implementation of the Commitments.

10. Evaluate practices, criteria and tools based on solid evidence and the state-of-the-art in research on research, and make data openly available for evidence gathering and research.