A Study on Building Efficient Zero-Shot Relation Extraction Models

2026-03-01

AI Summary

This work addresses two key challenges in zero-shot relation extraction: the need for on-the-fly encoding of entity pairs, which hinders large-scale precomputation, and the absence of a rejection mechanism, leading to poor robustness against irrelevant inputs. The study presents the first taxonomy of zero-shot relation extraction models and proposes a practical framework that integrates a single-pass inference architecture with an explicit rejection mechanism. It further conducts a systematic evaluation of several state-of-the-art models—including AlignRE—under realistic retrieval scenarios. Experimental results reveal that existing approaches generally suffer from insufficient robustness, whereas AlignRE consistently achieves superior performance across all metrics, establishing a new benchmark and offering a promising direction toward efficient and scalable zero-shot relation extraction.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Natural Language Processing: Information Extraction
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Zero-shot relation extraction aims to identify relations between entity mentions using textual descriptions of novel types (i.e., previously unseen) instead of labeled training examples. Previous works often rely on unrealistic assumptions: (1) pairs of mentions are often encoded directly in the input, which prevents offline pre-computation for large scale document database querying; (2) no rejection mechanism is introduced, biasing the evaluation when using these models in a retrieval scenario where some (and often most) inputs are irrelevant and must be ignored. In this work, we study the robustness of existing zero-shot relation extraction models when adapting them to a realistic extraction scenario. To this end, we introduce a typology of existing models, and propose several strategies to build single pass models and models with a rejection mechanism. We adapt several state-of-the-art tools, and compare them in this challenging setting, showing that no existing work is really robust to realistic assumptions, but overall AlignRE (Li et al., 2024) performs best along all criteria.

Problem

• zero-shot relation extraction
• realistic extraction scenario
• rejection mechanism
• offline pre-computation
• irrelevant inputs

Innovation

• zero-shot relation extraction
• rejection mechanism
• single-pass model
• realistic evaluation
• entity mention encoding