density-evolution analysis

This work investigates the phase transition behavior of belief propagation-guided elimination (BP-guided elimination) for solving random k-XORSAT. Using statistical physics–based message-passing analysis, random constraint satisfaction problem (CSP) phase transition theory, and analytic combinatorics coupled with large deviations methods, we derive—for the first time—the explicit critical density threshold at which the algorithm succeeds with constant probability Ω(1). We rigorously prove that the algorithm’s success–failure transition coincides precisely with the non-reconstruction/condensation phase transition occurring during the elimination process, thereby exactly locating both phase transitions. Our results not only provide the first rigorous validation of the long-standing physics conjecture linking algorithmic performance to structural phase transitions, but also substantially advance beyond prior partial characterizations—constituting a theoretical breakthrough over the ICALP’24 result.

This work addresses the issue that expectation propagation (EP) may generate non-integrable beliefs during iteration, leading to inference failure—particularly in Bayesian inference tasks such as generalized linear models. To resolve this, the authors propose two novel EP frameworks that enforce integrability of beliefs by constraining the belief space itself rather than imposing restrictive conditions on messages, thereby preserving message flexibility while rigorously guaranteeing belief integrability. These frameworks are derived from a constrained Bethe free energy optimization perspective, seamlessly integrating variational inference with message-passing mechanisms to yield stable iterative update rules. Experimental results demonstrate that the proposed methods effectively avoid non-integrable beliefs in signal recovery tasks under generalized linear models, significantly improving estimation accuracy and extending the applicability of EP to non-standard probabilistic models.

This work addresses the persistent distributional drift and eventual model collapse in recursive self-training diffusion models caused by premature truncation of the backward process. The study establishes, for the first time, that under this mechanism the model converges geometrically to a unique limiting distribution, which admits a closed-form expression as a Gaussian-smoothed mixture of the data distribution. Leveraging Hermite spectral decomposition, the authors provide a low-pass filtering interpretation, identifying the accumulation of high-frequency errors as the root cause of drift. To mitigate this, they propose a progressive annealing truncation strategy. Combining Wasserstein-2 stability analysis with Gaussian mixture modeling, both theoretical and empirical results—validated on synthetic Gaussian mixtures and CIFAR-10—demonstrate that higher-order mode errors decay more rapidly and confirm the efficacy of the proposed approach.

This work investigates the non-asymptotic behavior of approximate message passing (AMP) algorithms in high-dimensional generalized Gaussian random matrix models with arbitrary variance profiles, focusing on entrywise precise characterization under non-i.i.d. observations. We propose the first high-dimensional, non-asymptotic, leave-one-out (LOO) AMP representation framework, departing from classical low-dimensional state evolution paradigms. Leveraging LOO analysis, integration by parts, concentration inequalities, and inductive arguments, we rigorously derive the finite-sample distribution of ridge estimators under heterogeneous covariates and establish an explicit relationship between the vector-valued effective noise and regularization parameters—governed by a high-dimensional system of equations. Our key contribution is the first entrywise exact modeling of AMP under non-i.i.d. Gaussian designs, providing a unified non-asymptotic theoretical foundation for high-dimensional regularized estimation.

This work addresses smooth stochastic optimization of probability measures over compact subsets of Euclidean space, motivated by applications in emergency response and experimental design. We extend the Frank–Wolfe (FW) algorithm to the infinite-dimensional space of probability measures—its first such generalization—and prove that subproblem solutions are necessarily Dirac measures concentrated at extreme points. We propose a stochastic FW algorithm with Monte Carlo sampling (sFW) and establish rigorous convergence guarantees: for convex objectives, the optimization gap is (O(1/k)) almost surely and in expectation; for nonconvex objectives, the FW gap is (O(1/sqrt{k})); and with fixed step size and sample size, exponential convergence to an (varepsilon)-optimal solution is achieved. Additionally, we derive a central limit theorem for the objective value sequence. Our core contribution lies in systematically lifting the classical finite-dimensional FW framework to the infinite-dimensional space of probability measures, yielding a theoretically grounded and practically implementable stochastic optimization method.

This work addresses the lack of theoretical characterization of success mechanisms and reliability under multiple failure modes in multi-agent AI systems performing collaborative tasks. The authors model collaboration as message passing on a sparse, role-structured factor graph, introducing a set-valued propagation scheme that incorporates three types of erasure-like failures and integrates nonlinear, value-asymmetric logical verifiers. They extend density evolution theory—previously limited to linear settings—to agent networks featuring nonlinear verification and multiple failure modes, revealing an inherent asymmetry between positive and negative verification under logical operators such as AND. A new threshold and finite-length analysis framework is established for both deterministic and random graph sequences. The theory accurately predicts the asymptotic fraction of unresolved sub-propositions, recovers classical LDPC-BEC results in the XOR case, and uncovers asymmetric behavior of verification certificates in the AND case.

This work addresses the challenge that traditional discrete probability distributions rely on manually derived analytical forms, hindering the automatic discovery of interpretable models. We propose Symbolic Density Estimation (SDE), a novel framework that, for the first time, integrates structural priors, evolutionary search, and validity-aware parameter inference to automatically discover closed-form probability mass functions within a structured symbolic space composed of elementary mathematical operations. SDE accommodates complex distributional features such as zero-inflation and finite mixtures. We introduce the first systematic benchmark dataset for this task and demonstrate that SDE accurately recovers all target distribution families. On real-world data, SDE discovers concise, interpretable mixture models that achieve superior goodness-of-fit compared to standard methods.

This work addresses the computational inefficiency and limited probabilistic characterization inherent in traditional Monte Carlo methods, which rely on repeated pathwise simulations of stochastic iterative equations. The authors propose a unified framework that directly evolves the full probability density of the state vector, thereby circumventing the need for repeated sampling and enabling accurate treatment of nonlinear, discontinuous, and nonstandard stochastic systems. For the first time, this approach explicitly propagates the complete probability density through stochastic iterative maps, overcoming fundamental limitations of path-based simulation. The framework is extended to novel applications including global optimization under uncertainty and chaotic dynamical systems. Key contributions include an efficient method for simulating stochastic differential equations, the development of Full-Density Gradient Descent (FDGD) for global optimization in uncertain environments, and empirical validation of the approach on chaotic mappings.

This study investigates the asymptotic behavior of generalized first-order methods on structured—particularly deterministic—matrices, with the aim of constructing iterative algorithms that preserve conditional Gaussianity. By introducing the theory of limiting traffic distributions together with graph expansion techniques, the authors compute for the first time the traffic distribution of deterministic matrices such as Walsh–Hadamard ensembles. Leveraging this result, they propose a novel approximate message passing (AMP) algorithm that exhibits universal Gaussian dynamics across a broad class of matrices, and provide a combinatorial interpretation of its Onsager correction term. This work unifies and extends existing AMP variants, resolving a conjecture partially posed by Marinari et al. in 1994.