Projection Pursuit CPCANet for Domain Generalization

2026-07-24

AI Summary

This work addresses the challenge in domain generalization where small-batch training leads to rank-deficient covariance matrices, hindering stable extraction of domain-invariant features. To circumvent covariance estimation altogether, the authors propose a covariance-free framework that learns a globally orthogonal basis on the Stiefel manifold. This basis is jointly optimized with network parameters via the Cayley transform, and a symmetry-breaking, median-based projection pursuit objective is introduced to extract shared principal components. The method achieves state-of-the-art performance across four standard domain generalization benchmarks while significantly enhancing training stability.

Abstract

Domain Generalization (DG) aims to learn representations robust to distribution shifts. Recent geometric alignment methods, such as CPCANet, extract domain-invariant structures through batch-wise Common Principal Component Analysis (CPCA). However, CPCANet suffers from rank-deficient covariance estimation due to the small-sample-size issue in mini-batch training. To address this limitation, we propose Projection Pursuit CPCANet (PP-CPCANet), a covariance-free framework that learns a global orthogonal basis on the Stiefel manifold and jointly optimizes it with network parameters via the Cayley transform. We further introduce a symmetry-breaking detached-median PP dispersion objective to extract common principal components (CPCs) with dense and robust optimization signals. Experiments on four DG benchmarks show that PP-CPCANet achieves SOTA performance while maintaining stable training.

Problem

• Domain Generalization
• Covariance Estimation
• Small-Sample-Size Problem
• Common Principal Component Analysis
• Rank Deficiency

Innovation

• Projection Pursuit
• Covariance-free
• Stiefel manifold
• Cayley transform
• Domain Generalization