Domain Generalization: A Tale of Two ERMs

2025-10-05

AI Summary

Domain generalization (DG) aims to train models that generalize to unseen test domains. This paper identifies a critical disparity in empirical risk minimization (ERM) performance across distribution shift types: while ERM remains robust under covariate shift, it suffers severe degradation under posterior shift. To address this, we propose Domain-Aware Feature Enhancement (DAFE), a method that explicitly incorporates domain information into the ERM framework to mitigate posterior shift effects. Theoretical analysis establishes how domain-aware feature augmentation tightens the generalization error bound by reducing the discrepancy between domain-specific posterior distributions. Extensive multimodal experiments—spanning vision and language tasks—demonstrate that DAFE significantly outperforms standard ERM and state-of-the-art DG methods on datasets dominated by posterior shift. Our work introduces an interpretable, lightweight, and deployment-friendly DG paradigm, underscoring the necessity of shift-type diagnosis and shift-specific modeling in generalization-aware learning.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Adversarial Attacks & Robustness
• Application Domains: Misinformation & Fake News

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Domain generalization (DG) is the problem of generalizing from several distributions (or domains), for which labeled training data are available, to a new test domain for which no labeled data is available. A common finding in the DG literature is that it is difficult to outperform empirical risk minimization (ERM) on the pooled training data. In this work, we argue that this finding has primarily been reported for datasets satisfying a emph{covariate shift} assumption. When the dataset satisfies a emph{posterior drift} assumption instead, we show that ``domain-informed ERM,'' wherein feature vectors are augmented with domain-specific information, outperforms pooling ERM. These claims are supported by a theoretical framework and experiments on language and vision tasks.

Problem

• Addressing domain generalization under covariate shift conditions
• Proposing domain-informed ERM for posterior drift scenarios
• Validating method with theoretical framework and cross-domain experiments

Innovation

• Augmenting features with domain-specific information
• Addressing posterior drift instead of covariate shift
• Outperforming pooled ERM using domain-informed ERM