MMDG-Bench: A Benchmark for Multimodal Domain Generalization

2026-05-30

AI Summary

This work addresses the lack of a unified benchmark and systematic evaluation in multimodal domain generalization (MMDG), where existing research is largely confined to action recognition and lacks methodological coherence. To bridge this gap, we introduce MMDG-Bench, the first comprehensive benchmark for MMDG that spans multiple tasks including action recognition and face anti-spoofing. We further propose two principled frameworks: Domain-to-Multimodal (D2M), which applies domain generalization before multimodal learning, and Multimodal-to-Domain (M2D), which reverses this order. By integrating five representative domain generalization methods with diverse modalities—RGB, depth, infrared, audio, and optical flow—we instantiate ten reproducible baselines. Extensive experiments demonstrate that structured combinations within our frameworks significantly outperform current state-of-the-art approaches, reveal a strong correlation between framework choice and modality relationship stability, and confirm consistent performance gains from domain generalization across different backbones, with stronger backbones amplifying these benefits.

Abstract

Multi-modal Domain Generalization (MMDG) seeks to leverage complementary modalities to enhance model robustness on unseen domains. Despite extensive progress in Multi-modal Learning (MML) and Domain Generalization (DG) as individual fields, their systematic integration remains under-explored. Current MMDG research is largely confined to action recognition and lacks standardized evaluation protocols. To address this, we introduce MMDG-Bench, a comprehensive benchmark featuring two foundational frameworks: DG then MML (D2M) and MML then DG (M2D). We provide unified experimental protocols across diverse tasks, including video-audio-flow action recognition and RGB-Depth-IR face anti-spoofing. By instantiating ten MMDG baselines through pairing a unified MML configuration with five DG techniques under both D2M and M2D orderings, we demonstrate that these structured combinations frequently outperform existing state-of-the-art methods, underscoring the necessity of a unified benchmarking effort. Our analysis yields three key insights: (1) Integrating DG techniques provides consistent generalization gains across various backbones, whereas non-DG methods are highly sensitive to backbone shifts; (2) The optimal framework choice depends on inter-modal stability: D2M excels when modal relations are stable across domains, while M2D is more robust to cross-domain relational variance; (3) Stronger backbones yield amplified performance dividends when integrated into our structured frameworks. MMDG-Bench provides a principled foundation and actionable design guidelines for future research in multi-modal robustness. Code is released at https://github.com/qszhan/MMDG-Bench.

Problem

• Multimodal Domain Generalization
• Benchmark
• Evaluation Protocol
• Multi-modal Learning
• Domain Generalization

Innovation

• Multimodal Domain Generalization
• Benchmark
• Structured Framework
• Cross-domain Robustness
• Unified Evaluation Protocol