Differentiability and overlap concentration in optimal Bayesian inference

2025-01-15

AI Summary

This work addresses optimal Bayesian inference in finite-rank tensor product models, focusing on how the parametrized linear channel strength $h$ governs prediction accuracy and optimal guessing strategies. Methodologically, it integrates tools from statistical physics, large deviations theory, and variational analysis. The key contribution is the first rigorous establishment of equivalence among three fundamental properties: differentiability of the free energy, concentration of overlaps, and convergence of the minimum mean-square error (MMSE). Crucially, it proves that the free energy is smooth at all interior points in the low signal-to-noise ratio (SNR) regime—thereby establishing replica symmetry on a full-measure set without assuming strong replica symmetry. Moreover, it shows that $h = 0$ lies in the closure of the differentiability set and that the set of differentiability points has full Lebesgue measure for all SNRs, enabling deterministic characterization of both overlap structure and MMSE.

Technology Category

Machine Learning: Matrix & Tensor MethodsReasoning under Uncertainty: Relational Probabilistic ModelsCognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

Security and Privacy: Large-scale security measurementsGraph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphsSocial Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks

Abstract

In this short note, we consider models of optimal Bayesian inference of finite-rank tensor products. We add to the model a linear channel parametrized by $h$. We show that at every interior differentiable point $h$ of the free energy (associated with the model), the overlap concentrates at the gradient of the free energy and the minimum mean-square error converges to a related limit. In other words, the model is replica-symmetric at every differentiable point. At any signal-to-noise ratio, such points $h$ form a full-measure set (hence $h=0$ belongs to the closure of these points). For a sufficiently low signal-to-noise ratio, we show that every interior point is a differentiable point.

Problem

• Bayesian guessing
• parameter optimization
• signal strength variation

Innovation

• Bayesian inference
• tunable parameter
• adaptive accuracy improvement