Illuminating Visual Identity in Universal Multimodal Embeddings

2026-08-03

AI Summary

This work addresses the challenge that universal multimodal embeddings (UME) often lack effective visual identity discriminability, which hinders performance in tasks such as instance retrieval, re-identification, and identity-consistent content generation. To tackle this issue, we formally define and systematically study the problem of visual identity representation within UME for the first time, proposing a unified VisID modeling framework that jointly optimizes general multimodal and identity-specific representations through an identity-aware sampling mechanism. We further introduce MVEB, the first large-scale multimodal visual identity benchmark, encompassing both real-world and synthetic data, to facilitate training and evaluation. Extensive experiments demonstrate that our approach substantially enhances identity discriminability in UME while preserving strong general multimodal capabilities, confirming its effectiveness and generalization across diverse settings.

Abstract

Universal Multimodal Embeddings (UMEs) aim to unify various modalities and tasks into a shared representation space. In recent years, this field has witnessed substantial progress driven by the development of Multimodal Large Language Models (MLLMs). However, a crucial capability, visual identity discrimination, remains underexplored in existing UME methods, despite its critical role in a wide range of tasks, including instance retrieval, re-identification, and identity preservation in AI-generated content. To bridge this gap, we propose a unified formulation for visual identity discrimination~(VisID) and introduce $\textbf{MVEB}$ ($\textbf{M}$ultimodal $\textbf{V}$isual Identity $\textbf{E}$mbedding $\textbf{B}$enchmark), a large-scale benchmark curated from both real-world and synthetic datasets to support evaluation and training. Furthermore, we present a simple yet effective learning framework that jointly optimizes general multimodal and visual identity representations through a carefully designed identity-aware sampling mechanism. Extensive experiments demonstrate that our approach successfully endows UMEs with strong identity discrimination capability and maintains competitive general multimodal performance. We believe this work not only illuminates a critical yet neglected capability, but also takes a step toward more holistic universal multimodal embeddings. Code and data are available at \href{https://chrisclear3.github.io/MVEB}{MVEB}.

Problem

• visual identity discrimination
• universal multimodal embeddings
• instance retrieval
• re-identification
• identity preservation

Innovation

• Visual Identity Discrimination
• Universal Multimodal Embeddings
• Multimodal Benchmark
• Identity-Aware Sampling
• Multimodal Learning