Scientific production in the era of large language models.

2025-12-18

Science

AI Summary

This study systematically investigates the impact of large language models (LLMs) on scientific production patterns, paper quality, and scholarly evaluation. Leveraging a dataset comprising 2.1 million preprints, 28,000 peer-review reports, and 246 million citation accesses, and integrating scientometric analysis, natural language processing, and citation behavior modeling, the research reveals— for the first time—that LLM usage inverts the traditional positive correlation between linguistic complexity and paper quality. Findings indicate that authors using LLMs produce 23.7%–89.3% more papers, with increased linguistic sophistication but limited substantive contribution. Moreover, LLM-assisted authors significantly broaden their citation scope, preferentially citing more diverse, cutting-edge, yet lower-cited works, thereby reshaping the academic citation ecosystem.

Technology Category

Machine Learning: Large Multimodal Models (LMMs)Natural Language Processing: (Large) Language ModelsPlanning, Routing, and Scheduling: Planning with Language Models

Application Category

User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendationEconomics, Online Markets and Human Computation: Cost models of using LLMs in production systemsSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

With the production process rapidly evolving, science policy must consider how institutions could evolve.

Problem

• Large Language Models
• scientific production
• research evaluation
• manuscript quality
• citation diversity

Innovation

• Large Language Models
• scientific production
• manuscript drafting
• citation diversity
• research evaluation