LaME: Learning to Think in Latent Space for Multimodal Embedding via Information Bottleneck

2026-06-11

AI Summary

Existing multimodal embedding methods based on chain-of-thought (CoT) reasoning suffer from high computational overhead and reliance on high-quality CoT annotations, making them ill-suited for low-latency retrieval and large-scale training. This work proposes LaME, the first approach to enable efficient embedding-guided reasoning in latent space. LaME employs K learnable reasoning tokens to construct a fixed-capacity information bottleneck and leverages a two-stage weakly supervised training strategy to perform implicit reasoning within a single forward pass—eliminating the need for explicit CoT annotations. Evaluated on the MMEB-v2 and MRMR benchmarks, LaME matches or even surpasses the performance of explicit CoT models while achieving a 60× speedup in inference and throughput approaching that of discriminative embedding models.

Abstract

Reasoning-driven universal multimodal embedding has advanced rapidly by introducing Chain-of-Thought (CoT) reasoning into the embedding pipeline. Despite the strong performance across both general and complex tasks, this paradigm suffers from two core limitations: (i) autoregressive CoT reasoning incurs high computational cost, making it impractical for low-latency retrieval; and (ii) embedding performance is heavily coupled with CoT annotation quality, making large-scale training unreliable. These raise fundamental questions: Is textual CoT the optimal form of reasoning for embedding, and can effective embedding reasoning be accomplished in latent space? To this end, we propose LaME (Latent Reasoning Multimodal Embedding), which formulates embedding-oriented latent reasoning as a weakly supervised information bottleneck. LaME employs K learnable reason tokens as a fixed-capacity bottleneck, completing all reasoning within a single forward pass. The two weak supervision signals structurally decouple contrastive from autoregressive objectives and eliminate dependence on CoT annotations, while a two-stage training pipeline ensures stable convergence. Experiments on MMEB-v2 and MRMR show that LaME achieves competitive performance, surpassing some explicit CoT-based models, while delivering 60x faster inference than explicit CoT methods and 2x faster than latent baselines with throughput comparable to discriminative embedding models. Code will be released.

Problem

• multimodal embedding
• Chain-of-Thought reasoning
• latent space
• information bottleneck
• low-latency retrieval

Innovation

• Latent Reasoning
• Information Bottleneck
• Multimodal Embedding
• Chain-of-Thought
• Weak Supervision