Compressed Active Subspaces for Scalable Bayesian Inference

2026-09-16

AI Summary

该研究通过提出压缩活动子空间（CAS）方法，解决了高维模型中因存储大量全维度梯度而导致的内存限制问题，使大规模模型的贝叶斯推理成为可能。

Abstract

Active subspace methods provide a framework for quantifying predictive uncertainty in high-dimensional models by identifying and performing inference along parameter directions that have the greatest influence on the model output. However, the construction of active subspaces requires storing many full-dimensional model gradients, which becomes prohibitive as model size increases. We address this limitation by proposing Compressed Active Subspaces (CAS), a scalable approach that first maps the model parameters to a compressed space using a structured isometric embedding and then constructs the active subspace within this reduced parameterization. Our approach substantially reduces the memory required for active subspace construction and enables Bayesian inference for large models where standard active subspace methods become impractical. We demonstrate the scalability of CAS on neural networks of increasing size while maintaining predictive performance and robust uncertainty estimates.

Problem

• Active Subspaces
• Bayesian Inference
• High-dimensional Models
• Memory Consumption

Innovation

• Compressed Active Subspaces
• scalable Bayesian inference
• structured isometric embedding
• reduced parameterization