Technology Category
Machine Learning: Representation LearningComputer Vision: Representation Learning for VisionNatural Language Processing: Sentence-level Semantics, Textual Inference, etc.
Refine and Purify: Orthogonal Basis Optimization with Null-Space Denoising for Conditional Representation Learning
🤖 AI Summary
This work addresses the challenges of subspace basis sensitivity and cross-subspace interference in conditional representation learning by proposing an adaptive orthogonal semantic basis construction framework coupled with a null-space denoising projection mechanism. The method leverages singular value decomposition combined with curvature-based truncation to refine orthogonal bases, while projecting non-target semantics into the null space of irrelevant subspaces to effectively decouple desired signals from interfering components. This approach significantly enhances representation purity and generalization capability, achieving state-of-the-art performance across diverse downstream tasks including customized clustering, classification, and retrieval.
Technology Category
Application Category
📝 Abstract
Conditional representation learning aims to extract criterion-specific features for customized tasks. Recent studies project universal features onto the conditional feature subspace spanned by an LLM-generated text basis to obtain conditional representations. However, such methods face two key limitations: sensitivity to subspace basis and vulnerability to inter-subspace interference. To address these challenges, we propose OD-CRL, a novel framework integrating Adaptive Orthogonal Basis Optimization (AOBO) and Null-Space Denoising Projection (NSDP). Specifically, AOBO constructs orthogonal semantic bases via singular value decomposition with a curvature-based truncation. NSDP suppresses non-target semantic interference by projecting embeddings onto the null space of irrelevant subspaces. Extensive experiments conducted across customized clustering, customized classification, and customized retrieval tasks demonstrate that OD-CRL achieves a new state-of-the-art performance with superior generalization.
Problem
Research questions and friction points this paper is trying to address.
conditional representation learning
subspace basis sensitivity
inter-subspace interference
semantic denoising
orthogonal basis
Innovation
Methods, ideas, or system contributions that make the work stand out.
Orthogonal Basis Optimization
Null-Space Denoising
Conditional Representation Learning
Subspace Interference Suppression
Singular Value Decomposition
🔎 Similar Papers
No similar papers found.
