The disruption index does not measure scientific innovation

2026-06-05

AI Summary

This study critically examines the validity of the “disruptiveness index” as a metric for scientific innovation, particularly challenging its use in justifying policy recommendations such as mandatory retirement. Through bibliometric analysis and critical assessment of indicator design, the authors systematically interrogate the theoretical foundations, empirical evidence, and underlying data logic of the index, revealing a lack of rigorous support for its claims to accurately capture scientific novelty. The work highlights the pervasive risks associated with “black-box” metrics in research evaluation and underscores the necessity of subjecting quantified policy tools to stringent validation. By exposing the limitations of an influential yet inadequately scrutinized indicator, this research serves as a timely caution against the uncritical adoption of unverified metrics and offers methodological guidance for developing more reliable approaches to assessing scientific impact.

Abstract

A paper recently published in Science under the rubric of Policy Article argued that what the authors call scientific disruption declines with academic age, and that this decline is related to the absence of mandatory retirement for older academics. Since its publication, its conclusions and policy suggestions in relation to mandatory retirement have received considerable media attention. Thus, it is worth taking a closer look at the proposed measure of disruption since all the analysis and conclusions are based on the results obtained from this index, thus taking it as valid. The issues we address are not specific to this article and can be found in many papers using bibliometric data that propose a new index on the basis of common sense intuition and then using it as a black boxed instrument to measure quality, innovation or, now, disruption for creating rankings and formulate policy actions on the basis of the calculated values of the index.

Problem

• disruption index
• scientific innovation
• bibliometric data
• research evaluation
• policy formulation

Innovation

• disruption index
• scientific innovation
• bibliometrics
• research evaluation
• policy implications