Citations Are Late: Reading epistemic instability from what papers believe, years before the citation graph catches up

2026-10-05

AI Summary

This study addresses the challenge of delayed identification of scientific paradigm shifts caused by the inherent lag in citation-based metrics. To overcome this limitation, we propose an early warning mechanism grounded in the content of paper abstracts. By analyzing the temporal evolution of modeling belief distributions within abstracts, we construct a content-level cognitive instability signal capable of prospectively predicting paradigm transitions. Applied to the natural language processing domain, this signal detects the shift from RNNs to Transformers approximately 25 quarters earlier than the CD5 citation metric. Supported by preregistered experimental validation, our results demonstrate that the proposed content signal achieves significantly higher detection accuracy than keyword- and embedding-based approaches, effectively revealing cross-domain trajectories of cognitive evolution.

Abstract

Paradigm shifts in science are visible in what researchers assert and contest before they are visible in the citation graph. We ask whether a cheap, content-level signal of epistemic instability, derived from the changing distribution of stated modelling beliefs in paper abstracts, can anticipate a paradigm shift earlier than the dominant citation-based disruption index (CD5). On the displacement of recurrent networks by Transformers in NLP, a pre-registered content signal crosses its detection threshold in 2016-Q1, whereas a real-time CD5 monitor cannot even observe the 2017 breakthrough until 2022-Q2, since CD5 needs a five-year forward-citation window: a lead of about 25 quarters. The flat citation baseline is not an artifact of one index, as our CD5, a reference-normalised variant, and an authoritative precomputed index all sit near zero across the shift. The lead is also not a faster proxy: at equal latency the signal beats the content competitors tested, including a learned CD-from-text model and an embedding disruption measure, and the decomposition sees what a keyword cannot (keyword-blind AUC of about 0.9, reproduced on human labels). The lead over CD5 is an observability lead. Which signal carries it depends on the shift: belief adoption in NLP, contestation and applicability stress in vision. Measured against the breakthroughs themselves, the signal leads by five quarters in NLP and is contemporaneous in computer vision.

Problem

• paradigm shift
• epistemic instability
• citation graph
• disruption index
• early detection

Innovation

• epistemic instability
• paradigm shift detection
• content-level signal
• citation disruption index
• modelling beliefs