Screening articles by citation reputation

2025-02-01

Quantitative Science Studies

AI Summary

Existing citation metrics conflate paper popularity with genuine scholarly impact and remain vulnerable to non-academic citation motives. To address this, we propose Reliable Citation (RC), the first framework integrating institutional international rankings (e.g., QS, ARWU) and peer-recognized reputation into citation analysis. RC decouples raw citation counts from substantive academic contribution via institutional-reputation weighting and citation-motive classification. The method synthesizes Web of Science citation data, Clarivate InCites journal-level metrics, and multi-source institutional weights to construct a hierarchical evaluation system. Empirical validation in mathematics demonstrates that scholars ranked highly by RC significantly outperform those identified solely by conventional citation counts; moreover, RC effectively detects papers whose citations deviate from substantive innovation—thereby enhancing precision in literature screening and improving the credibility of scholarly impact assessment.

Technology Category

Reasoning under Uncertainty: Other Foundations of Reasoning under UncertaintyKnowledge Representation and Reasoning: Qualitative ReasoningMachine Learning: Learning Preferences or Rankings

Application Category

Search and Retrieval-Augmented AI: Web evaluation methodologies and metricsWeb Mining and Content Analysis: Web data provenance, reliability, and authenticityUser Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

We introduce reputable citations (RC), a method to screen and segment a collection of papers by decoupling popularity and influence. We demonstrate RC using recent works published in a large set of mathematics journals from Clarivate’s Incites Essential Science Indicators, leveraging Clarivate’s Web of Science for citation reports and assigning prestige values to institutions based on well-known international rankings. We compare researchers drawn from two samples: highly cited researchers (HC) and mathematicians whose influence is acknowledged by peers (Control). RC scores distinguish the influence of researchers beyond citations, revealing highly cited mathematical work of modest influence. The control group, comprising peer-acknowledged researchers, dominates the top tier of RC scores despite having fewer total citations than the HC group. Influence, as recognized by peers, does not always correlate with high citation counts, and RC scores offer a nuanced distinction between the two. With development, RC scores could automate screening of citations to identify exceptional and influential research, while addressing manipulative practices. The first application of RC reveals mathematics works that may be cited for reasons unrelated to genuine research advancements, suggesting a need for continued development of this method to mitigate such trends. https://www.webofscience.com/api/gateway/wos/peer-review/10.1162/qss_a_00355

Problem

• Decouple popularity and influence
• Distinguish researcher influence beyond citations
• Automate screening to identify influential research

Innovation

• Reputable Citations for influence screening
• Decoupling popularity from influence metrics
• Automated citation screening for exceptional research