On the Information Bottleneck for Stable Variables

2026-10-06

AI Summary

This study investigates the Gaussian information bottleneck generalized to jointly stable random variables. Focusing on the additive model X = Y + A, it establishes, for the first time, a theoretical framework for the linear information bottleneck with stable variables. By integrating information theory, probability statistics, and optimization theory, closed-form solutions and critical values for stochastic linear encoders are derived. The analysis demonstrates that such encoders are strictly suboptimal in non-Gaussian settings yet asymptotically optimal under high compression rates. This work not only successfully recovers classical Gaussian information bottleneck results but also reveals optimality boundaries in non-Gaussian scenarios, thereby providing a rigorous theoretical foundation for information compression under stable distributions.

Abstract

We consider a generalization of the Gaussian information bottleneck problem to jointly stable variables $X$ and $Y$. Specifically, for $X = Y + A$, where $A$ is stable and independent of $Y$, we find optimal stochastic linear encoders of the form $T = kX + N$, $N$ being stable and independent of $X$. We characterize the linear stable information bottleneck in closed-form and its critical value. Furthermore, we show that, except for the Gaussian case, such stochastic linear encoders are strictly suboptimal, however asymptotically optimal at large compression rate. Our linear solution recovers the Gaussian information bottleneck for jointly Gaussian variables.

Problem

• Information Bottleneck
• Stable Variables
• Linear Encoders
• Gaussian Information Bottleneck

Innovation

• Information Bottleneck
• Stable Variables
• Linear Encoder
• Asymptotic Optimality
• Gaussian Generalization