LA-RL: Label-Aware Self-Reflection for Reinforcement Learning in Information Extraction

2026-07-25

AI Summary

Current large language models struggle to precisely identify specific errors—such as label misassignment or boundary shifts—in structured information extraction outputs, limiting their self-correction capabilities. This work proposes the LA-RL framework, which for the first time integrates fine-grained, task-oriented diagnostic labels into a reinforcement learning–driven self-reflection process: the model first generates an extraction output, then diagnoses its error type, and conditionally revises the prediction accordingly, all without requiring a process reward model. Employing a single backbone architecture, the approach combines supervised fine-tuning with a two-stage GRPO reinforcement learning phase, leveraging diagnostic annotations generated by a teacher model for cold-start training. It substantially outperforms SFT baselines across named entity recognition, relation extraction, and event extraction tasks, achieving an average F1 of 6.83 on SciER, approximately 20 F1 points gain on out-of-distribution relation extraction, and improvements of 14.80 and 17.50 F1 points on trigger and argument extraction in DuEE1.0, respectively.

Abstract

Large language models show strong promise for information extraction (IE), but existing reflection-based correction methods are often misaligned with structured extraction outputs. Free-form self-reflection can flag an error, yet it rarely identifies whether the failure is a missing span, wrong label, boundary mismatch, invalid relation type, or reversed argument order. We introduce LA-RL (Label-Aware Reflective Reinforcement Learning), an outcome-supervised framework that guides IE self-correction with task-grounded diagnostic labels. A single backbone first predicts an extraction, diagnoses task-specific error labels, and then revises its output conditioned on the diagnosis. Training starts from diagnostic data labeled by an annotation model for cold-start supervised fine-tuning and proceeds through two GRPO stages that reward final extraction quality, format validity, and first-pass correctness, without a process reward model. Experiments on named entity recognition, relation extraction, and event extraction show consistent same-backbone gains over SFT, including 6.83 average F1 on SciER relation extraction, about 20 F1 on out-of-distribution relation extraction, and 14.80 trigger F1 plus 17.50 argument F1 on DuEE1.0. Ablations show that reflection structure is task-sensitive: stronger constraints benefit relation extraction, whereas named entity recognition needs less restrictive correction under domain shift.

Problem

• information extraction
• self-reflection
• structured output
• error diagnosis
• large language models

Innovation

• Label-Aware Reflection
• Reinforcement Learning
• Information Extraction
• Error Diagnosis
• Outcome-Supervised Learning