Can Circuit Alignment Predict OOD Generalization?

2026-09-25

AI Summary

This study addresses the inability of existing weight-based metrics to predict out-of-distribution (OOD) generalization. We propose the Circuit Alignment Score, which leverages graph kernels to compare class-specific circuits across domains, enabling OOD performance prediction without access to target data. Methodologically, by integrating Lebesgue integral modeling with Monte Carlo estimation, we construct the first provably consistent distributional robustness predictor, thereby bridging a theoretical gap in structure-aware rerouting. Experimental evaluations demonstrate that our approach achieves a rank correlation of 0.88 on the PACS benchmark, significantly outperforming baselines such as Centered Kernel Alignment (CKA), while empirically validating the theoretically established convergence rates.

Abstract

Can out-of-distribution (OOD) generalization be predicted from a trained model's weights alone, without any target-domain data? Existing representational similarity metrics (CKA, SVCCA, RSA) compare activations rather than forecast generalization. We show they are provably insensitive to structural rerouting in the computational graph, the very change distribution shift induces. We close this gap with the Circuit Alignment Score (CAS), which compares class-specific circuits across domains via graph kernels, decomposed into same-class coherence and cross-class confusion. Casting CAS as a Lebesgue integral over the domain distribution, we prove its Monte Carlo estimate recovers the ground-truth ranking of learners by OOD accuracy, with pairwise inversion error vanishing at rate $O(1/M)$, where $M$ is the number of sampled domains. Across $48$ learners on PACS, CAS attains $0.88$ rank correlation with OOD accuracy, versus $0.58$ (CKA), $0.23$ (SVCCA), and $0.14$ (RSA), with similar trends on other benchmarks and even against data-dependent methods, making it the first provably consistent predictor of distributional robustness requiring neither target-domain data nor labels. The code is available at: https://github.com/ayanban011/ACE

Problem

• Out-of-distribution generalization
• Circuit alignment
• Representational similarity
• Distribution shift
• Predictive metrics

Innovation

• Out-of-distribution generalization
• Circuit Alignment Score
• Graph kernels
• Representational similarity
• Distributional robustness