information bottleneck fusion

This study addresses the limitation of the standard Information Bottleneck (IB) in disentangling label-relevant structures from irrelevant noise, which renders models prone to overfitting in few-shot scenarios. Building upon a label-induced partitioned reconstruction IB, this work proposes a dual-bottleneck framework that independently regulates global capacity and intra-conditional information. By achieving an exact decomposition of the conditional KL divergence and introducing a simplex structural prior to constrain latent space geometry, the method effectively disentangles noise. This approach integrates IB theory, structured latent variable modeling, and deep learning regularization techniques. It yields substantial improvements on low-data classification tasks while maintaining consistent performance gains across dense prediction benchmarks.

This work addresses lossy image compression for multi-task scenarios, proposing a framework that jointly optimizes reconstruction fidelity, perceptual quality, and classification accuracy. We establish, for the first time, an information-theoretic rate–distortion–classification (RDC/RPC) triadic model and derive its closed-form solution. Theoretically, we prove that under RPC constraints, classification performance and perceptual fidelity are not fundamentally trade-offs, and reveal the critical regulatory role of source noise in task-oriented compression. Leveraging the information bottleneck principle, we unify generative and discriminative objectives and derive optimal rate bounds for binary and Gaussian sources. Experiments demonstrate that our deep compression network achieves Pareto-optimality across PSNR, LPIPS, and classification accuracy—providing both theoretical foundations and a practical paradigm for task-driven compression.

Deep neural networks lack biologically plausible selective attention mechanisms, limiting both efficiency and accuracy in image recognition. To address this, we propose a spatial attention module grounded in information bottleneck theory. Our method explicitly optimizes mutual information: it minimizes the mutual information between the attention representation and the input to suppress redundancy, while maximizing the mutual information between the attention representation and task labels to enhance discriminability. Crucially, we introduce learnable anchors to quantize continuous attention scores—a novel design that strengthens information constraints and improves interpretability of attention maps. By integrating variational attention modeling with deep network embedding, our approach achieves significant performance gains across image classification, fine-grained recognition, and cross-domain classification tasks. The resulting attention maps exhibit high discriminability, strong background suppression, and enhanced interpretability.

This paper addresses the lack of a unified theoretical framework for variational dimensionality reduction. It proposes a unified Variational Information Bottleneck (VIB) framework that jointly optimizes encoder-based information compression and decoder-based generative fidelity, enabling principled information trade-offs in latent space. Key contributions include: (1) introducing DVSIB and beta-DVCCA—novel methods that extend the multivariate information bottleneck to deep variational settings for the first time; (2) establishing theoretical connections between DSIB and contrastive learning approaches (e.g., Barlow Twins) via mutual information regularization; and (3) proposing symmetric and weighted mutual information regularization to support multi-view representation learning and generative modeling. Evaluated on Noisy MNIST and CIFAR-100, the framework achieves significant improvements in classification accuracy, latent dimension efficiency, and sample efficiency, attaining state-of-the-art or superior performance.

To address low transmission efficiency and resource waste in edge video collaborative perception caused by channel constraints and spatiotemporal redundancy, this paper proposes the Priority Information Bottleneck (PIB) framework. PIB jointly models signal-to-noise ratio (SNR) and region-of-interest (RoI) coverage as a compact feature selection criterion, enabling reconstruction-free, low-latency feature transmission. It introduces an adaptive gating mechanism based on distributed online learning (DOL), with theoretical guarantees of asymptotic optimality—achieving a sublinear regret bound. Furthermore, PIB integrates deterministic information bottleneck principles with variational approximation for optimization. Extensive experiments across three heterogeneous edge devices demonstrate that, compared to five state-of-the-art codecs, PIB improves mean object detection accuracy (MODA) while reducing communication overhead by 82.65%, and maintains low latency even under weak channel conditions.

该论文通过互信息、信息瓶颈和部分信息分解等方法，从信息论角度统一理解多模态学习中的依赖、压缩与协同问题。

This study addresses the challenge of preserving predictive information about a target variable while removing irrelevant redundancy in data compression. Building on statistical decision theory, the authors propose an ℋ-mutual information framework that satisfies conditional independence (CV) and average generalization (AVG) criteria. They establish, for the first time, an equivalence between the generalized information bottleneck problem and Expected Sample Information (ESI), thereby enabling a computable characterization of a representation’s predictive utility. An alternating optimization algorithm is further developed to efficiently approximate the Pareto frontier between compression and utility. This work extends the applicability of classical mutual information and offers a new paradigm for information bottleneck theory that balances theoretical rigor with practical utility.

This study addresses the limitation of the classical information bottleneck in directly characterizing downstream decision errors by investigating the Chernoff bottleneck under mutual information constraints, aiming to maximize the error exponent of binary hypothesis testing under rate-limited conditions. Theoretically, it reveals the non-concavity of the Chernoff bottleneck curve and proves that optimality is achievable with only k+1 outputs. Algorithmically, an alternating optimization framework guaranteeing convergence and feasibility is proposed, integrating a generalized Blahut-Arimoto algorithm with nonlinear iterations for joint solution. Experiments on the 20 Newsgroups dataset demonstrate that compressing information while retaining merely 17% of its entropy preserves 90% of the error exponent and achieves near-lossless classification accuracy.

This work addresses the challenge of cloud removal in multimodal remote sensing imagery, where existing SAR-optical fusion methods often introduce speckle noise and cause excessive smoothing. To overcome these limitations, the authors propose the IB-HFN network, which employs a dual-stream backbone to preserve modality-specific features. Channel-wise variational information bottleneck is applied to compress SAR features and suppress noise, while a local-global gating mechanism safeguards optical details. A spatial information bottleneck fusion module, coupled with Dirac-initialized skip connections, enables decoupled optimization of noise suppression and texture preservation. The framework further integrates feature-level regularization with image-level multi-constraint objectives for joint optimization. Evaluated on the SEN12MS-CR dataset, the proposed method significantly outperforms state-of-the-art approaches in both structural integrity and spectral fidelity.

This study addresses the long-standing limitation in the Information Bottleneck (IB) framework, where the cardinality bound for optimal representations of binary sources has been constrained by generic upper bounds, thereby hindering computational efficiency. By exploiting the structural properties specific to the binary case, this work employs a separating hyperplane argument combined with concavity analysis of the ratio of second derivatives of entropy functions to transcend traditional generic bounds. It rigorously proves that the optimal representation for a binary source is itself binary, tightening the classical cardinality bound to the exact limit |U|≤|X|. This contribution not only establishes a theoretically optimal bound but also substantially reduces the computational complexity of solving IB problems.