Large Language Models for Citation Function Classification

2026-07-20

AI Summary

This study addresses the problem of citation function classification to deepen the understanding of semantic relationships between academic papers and advance bibliometric analysis. The authors introduce AC3, a novel dataset featuring fine-grained annotations across seven distinct citation categories, which explicitly differentiates neutral acknowledgments from evaluative citations. For the first time, the paper systematically evaluates state-of-the-art large language models—including Mistral 7B, Orca 2-7B, LLaMA 3.1-8B, Falcon 7B, and SciBERT—under zero-shot, few-shot, and fine-tuned settings. Experimental results demonstrate that fine-tuned Falcon 7B achieves a macro F1 score of 73.3% on ACL-ARC, significantly outperforming existing approaches and thereby validating both the effectiveness of the proposed framework and the research utility of the AC3 dataset.

Abstract

Citation function classification plays a crucial role in understanding the relationships between scientific publications and advancing bibliometric analysis. This study presents one of the first comprehensive evaluations of multiple state-of-the-art (SOTA) large language models (LLMs) for citation function classification, achieving new SOTA results on the ACL-ARC dataset. We systematically compare five models (Mistral 7B, Orca 2-7B, LLaMA 3.1-8B, Falcon 7B, and SciBERT) across zero-shot, few-shot, and fine-tuning approaches. Our fine-tuned Falcon 7B model achieves a 73.3% macro F1 score on ACL-ARC, representing a significant improvement over previous methods. Additionally, we introduce AC3, a novel dataset featuring a seven-category annotation scheme that distinguishes between neutral acknowledgments and explicit evaluative stances (more opinion-oriented citations - criticizing, complimenting, contradicting). The dataset is implemented across four context extraction variants to systematically evaluate the impact of contextual scope on classification performance. We also provide detailed analysis of model performance, experimental configurations, and limitations to guide future research in this domain. To our knowledge, this is one of the first studies dedicated to comprehensive model comparison for citation function classification, addressing a gap identified in recent surveys.

Problem

• citation function classification
• large language models
• bibliometric analysis
• scientific publications
• annotation scheme

Innovation

• citation function classification
• large language models
• Falcon 7B
• AC3 dataset
• contextual scope