Soft Contextualized Encoder For User Defined Text Classification

2026-01-06

arXiv.org

AI Summary

This work addresses the challenge of zero-shot generalization to unseen categories in user-defined text classification by proposing a soft contextualized encoder architecture. The approach introduces, for the first time, a soft contextualization mechanism that jointly models static soft prompt representations of candidate labels, label sets, and input queries, thereby enabling strong generalization to arbitrary new category sets. By integrating a context-aware encoder with a multi-source heterogeneous data training strategy, the model achieves state-of-the-art performance across multiple user-defined text classification benchmarks, significantly outperforming or matching existing baseline methods.

Technology Category

• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

User-Defined Text Classification (UDTC) considers the challenge of classifying input text to user-specified, previously unseen classes, a setting that arises frequently in real-world applications such as enterprise analytics, content moderation, and domain-specific information retrieval. We propose a soft-contextualized encoder architecture for UDTC which contextualizes each candidate label with the label set and a static soft prompt representation of the input query. Training on diverse, multi-source datasets enables the model to generalize effectively to zero-shot classification over entirely unseen topic sets drawn from arbitrary domains. We evaluate the proposed architecture both on held-out in-distribution test data and on multiple unseen UDTC benchmarks. Across datasets, the model achieves state-of-the-art performance, consistently outperforming or matching the baselines.

Problem

• User-Defined Text Classification
• Zero-shot Classification
• Unseen Classes
• Text Classification
• Domain Generalization

Innovation

• soft contextualized encoder
• user-defined text classification
• zero-shot classification
• static soft prompt
• label contextualization