Problem
Research questions and friction points this paper is trying to address.
Partial Information Decomposition
Redundancy
Synergy
Multivariate Gaussians
Blackwell order
Redundancy and synergy in multivariate Gaussians via the Blackwell order
🤖 AI Summary
This study addresses the challenge of defining and computing Partial Information Decomposition (PID) in high-dimensional continuous systems. By leveraging the Blackwell order, this work constructs a PID framework for multivariate Gaussian systems, establishing Blackwell redundancy as the unique measure satisfying natural axioms. The optimality of Gaussian channels is proven with a geometric interpretation provided. Furthermore, closed-form expressions and efficient numerical optimization algorithms are derived, enabling scalable computation to thousand-dimensional scales. The proposed method is successfully applied to optimal control problems, precisely identifying redundant and synergistic interactions between sensors and memory, thereby validating its theoretical effectiveness.
📝 Abstract
The goal of the partial information decomposition (PID) is to quantify the redundant and synergistic information that multiple sources provide about a target. PID has many applications in machine learning, neuroscience, and other fields, but defining and computing it for high-dimensional continuous systems remains challenging. Here, we define a PID for multivariate Gaussian systems based on the Blackwell order, which formalizes when one channel is more informative than another. We prove that Gaussian channels are optimal for extracting both redundant and union information, yielding an intuitive geometric interpretation and an efficient numerical algorithm for the PID. Our union information and synergy coincide with the well-known BROJA measures, and we derive closed-form expressions for both in the case of two sources. We also argue that Blackwell redundancy (which differs from BROJA) is the only existing redundancy measure that satisfies a set of natural desiderata. We demonstrate the scalability of our method on systems with up to a thousand dimensions or sources. Our approach is illustrated on an optimal control problem, where it identifies redundant and synergistic interactions between sensor and memory.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Partial Information Decomposition
Blackwell order
Multivariate Gaussians
Redundancy and Synergy
Scalability
🔎 Similar Papers
No similar papers found.
