Citation accuracy, citation noise, and citation bias: A foundation of citation analysis

2025-08-18

AI Summary

Citation noise—non-academic variation in citation decisions—seriously undermines the reliability of citation analysis, yet has long lacked systematic conceptualization and theoretical integration. This study first distinguishes two types: citation-level noise (arising from individual citing acts) and citation-pattern noise (emerging from aggregate disciplinary or institutional practices), clarifies their distinct interference mechanisms in knowledge flow assessment, and conceptually disentangles them from citation accuracy and citation bias, thereby establishing a novel theoretical framework for citation analysis. Through conceptual analysis and theoretical modeling grounded in bibliometrics and research evaluation theory, the study identifies multiple sources of non-academic variation in citation decisions. It proposes actionable noise-mitigation strategies and advocates their integration into reform initiatives such as CoARA, significantly enhancing the validity, reliability, and legitimacy of citation-based metrics in research evaluation.

Technology Category

• Reasoning under Uncertainty: Causality
• Data Mining & Knowledge Management: Other Foundations of Data Mining & Knowledge Management
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Citation analysis is widely used in research evaluation to assess the impact of scientific papers. These analyses rest on the assumption that citation decisions by authors are accurate, representing flow of knowledge from cited to citing papers. However, in practice, researchers often cite for reasons other than attributing intellectual credit to previous research. Citations made for rhetorical reasons or without reading the cited work compromise the value of citations as instrument for research evaluation. Past research on threats to the accuracy of citations has mainly focused on citation bias as the primary concern. In this paper, we argue that citation noise - the undesirable variance in citation decisions - represents an equally critical but underexplored challenge in citation analysis. We define and differentiate two types of citation noise: citation level noise and citation pattern noise. Each type of noise is described in terms of how it arises and the specific ways it can undermine the validity of citation-based research assessments. By conceptually differing citation noise from citation accuracy and citation bias, we propose a framework for the foundation of citation analysis. We discuss strategies and interventions to minimize citation noise, aiming to improve the reliability and validity of citation analysis in research evaluation. We recommend that the current professional reform movement in research evaluation such as the Coalition for Advancing Research Assessment (CoARA) pick up these strategies and interventions as an additional building block for careful, responsible use of bibliometric indicators in research evaluation.

Problem

• Citation noise undermines citation analysis validity
• Differentiating citation noise from bias and accuracy
• Proposing strategies to minimize noise in citations

Innovation

• Differentiates citation noise from bias
• Proposes framework for citation analysis
• Recommends strategies to minimize noise