Fane at SemEval-2025 Task 10: Zero-Shot Entity Framing with Large Language Models

2025-04-29

AI Summary

This study addresses zero-shot identification of entity framing roles in news narratives, aiming to uncover how media shape public perception of events through framing. We propose a hierarchical zero-shot classification paradigm: first identifying macro-level framing types (e.g., “attribution of responsibility”, “moral evaluation”), then refining to fine-grained entity roles (e.g., “agent”, “victim”), with context-sensitive prompting strategies and input optimization tailored to each level. Our approach integrates large language models’ semantic understanding with task-driven prompt engineering. Evaluated on SemEval-2025 Task 10, it achieves 89.4% accuracy on primary roles and 34.5% exact match rate—substantially outperforming end-to-end single-step baselines. The core contributions are: (1) empirical validation of hierarchical decomposition for zero-shot framing analysis; and (2) a transferable, context-adaptive prompting framework that generalizes across framing tasks and domains.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Knowledge Representation and Reasoning: Preferences
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Understanding how news narratives frame entities is crucial for studying media's impact on societal perceptions of events. In this paper, we evaluate the zero-shot capabilities of large language models (LLMs) in classifying framing roles. Through systematic experimentation, we assess the effects of input context, prompting strategies, and task decomposition. Our findings show that a hierarchical approach of first identifying broad roles and then fine-grained roles, outperforms single-step classification. We also demonstrate that optimal input contexts and prompts vary across task levels, highlighting the need for subtask-specific strategies. We achieve a Main Role Accuracy of 89.4% and an Exact Match Ratio of 34.5%, demonstrating the effectiveness of our approach. Our findings emphasize the importance of tailored prompt design and input context optimization for improving LLM performance in entity framing.

Problem

• Evaluating LLMs for zero-shot entity framing classification
• Assessing impact of context and prompts on framing accuracy
• Optimizing hierarchical task decomposition for role identification

Innovation

• Zero-shot classification using large language models
• Hierarchical approach for role identification
• Optimized input context and prompt strategies