Bridging Generative and Discriminative Learning: Few-Shot Relation Extraction via Two-Stage Knowledge-Guided Pre-training

2025-05-18

AI Summary

To address the dual challenges of data scarcity and poor generalization in few-shot relation extraction (FSRE), this paper proposes a two-stage collaborative framework integrating generative and discriminative modeling. First, it introduces an explanation-driven knowledge generation mechanism coupled with pattern-constrained synthetic data construction to enhance relational semantic coverage. Second, it devises a novel two-stage pretraining paradigm comprising masked span language modeling (MSLM) and span-level contrastive learning (SCL), explicitly strengthening relational structural awareness and discriminative capability. Crucially, the method avoids fine-tuning large language models (LLMs), achieving efficient knowledge transfer solely through prompt enhancement. Evaluated on multiple standard FSRE benchmarks, our approach establishes new state-of-the-art performance, significantly improving accuracy and robustness in low-resource settings. Experimental results validate the effectiveness of synergistically combining generative guidance with discriminative optimization for FSRE.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Statistical Relational/Logic Learning
• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Few-Shot Relation Extraction (FSRE) remains a challenging task due to the scarcity of annotated data and the limited generalization capabilities of existing models. Although large language models (LLMs) have demonstrated potential in FSRE through in-context learning (ICL), their general-purpose training objectives often result in suboptimal performance for task-specific relation extraction. To overcome these challenges, we propose TKRE (Two-Stage Knowledge-Guided Pre-training for Relation Extraction), a novel framework that synergistically integrates LLMs with traditional relation extraction models, bridging generative and discriminative learning paradigms. TKRE introduces two key innovations: (1) leveraging LLMs to generate explanation-driven knowledge and schema-constrained synthetic data, addressing the issue of data scarcity; and (2) a two-stage pre-training strategy combining Masked Span Language Modeling (MSLM) and Span-Level Contrastive Learning (SCL) to enhance relational reasoning and generalization. Together, these components enable TKRE to effectively tackle FSRE tasks. Comprehensive experiments on benchmark datasets demonstrate the efficacy of TKRE, achieving new state-of-the-art performance in FSRE and underscoring its potential for broader application in low-resource scenarios. footnote{The code and data are released on https://github.com/UESTC-GQJ/TKRE.

Problem

• Addressing data scarcity in Few-Shot Relation Extraction
• Bridging generative and discriminative learning paradigms
• Enhancing relational reasoning with two-stage pre-training

Innovation

• Generates explanation-driven knowledge and synthetic data
• Uses two-stage pre-training with MSLM and SCL
• Integrates LLMs with traditional relation extraction models