LTG at SemEval-2025 Task 10: Optimizing Context for Classification of Narrative Roles

2025-06-06

AI Summary

Existing masked language models (MLMs) struggle with narrative role classification in long documents due to their limited context windows, hindering effective capture of discriminative contextual cues. Method: We propose a lightweight, entity-centric heuristic context selection framework that dynamically identifies salient text segments via entity localization rules, and further enhances information density through context truncation and semantic reweighting. This enables efficient supervised fine-tuning on compact MLMs such as XLM-RoBERTa. Results: Our approach achieves 89.7% accuracy on the entity-frame classification subtask (Subtask 1) of SemEval-2025 Task 10—matching or exceeding the performance of large generative models (e.g., GPT-4, Claude-3) while reducing inference cost by two orders of magnitude. The core contribution is the first integration of entity-driven context refinement into MLM-based long-document narrative understanding, offering a high-efficiency, resource-frugal solution for constrained deployment scenarios.

Technology Category

• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Our contribution to the SemEval 2025 shared task 10, subtask 1 on entity framing, tackles the challenge of providing the necessary segments from longer documents as context for classification with a masked language model. We show that a simple entity-oriented heuristics for context selection can enable text classification using models with limited context window. Our context selection approach and the XLM-RoBERTa language model is on par with, or outperforms, Supervised Fine-Tuning with larger generative language models.

Problem

• Optimizing context selection for narrative role classification
• Enabling classification with limited context window models
• Outperforming larger models with efficient context heuristics

Innovation

• Entity-oriented heuristics for context selection
• XLM-RoBERTa model with limited context
• Outperforms larger generative language models