maximum-entropy projection

This work addresses a long-standing open problem in information theory: the lack of a general constructive solution for minimizing the index of coincidence of a joint distribution under given marginal constraints. The study provides the first complete characterization of the structural properties of optimal couplings, revealing that their zero entries exhibit a monotone staircase pattern. Building on this insight, the authors propose an explicit iterative construction algorithm that provably converges in finitely many steps to the global optimum for any feasible pair of marginals. The approach integrates tools from probabilistic coupling theory, combinatorial optimization, and iterative construction techniques, complemented by an asymptotic analysis of the measure of high-dimensional feasible sets, thereby resolving the constructive solvability of the index-of-coincidence minimization problem.

This work investigates the dual representation of the minimum f-divergence problem under integral constraints, a formulation with broad applications in sequential inference, multi-armed bandits, and distributionally robust optimization. Focusing on the observation space $[0,1]^K$, the authors propose a two-stage approach: first deriving a finite-dimensional convex dual for distributions with finite support, then extending the result to arbitrary distributions via an abstract interchange argument. This framework generalizes the well-known one-dimensional KL_inf duality under mean constraints to higher dimensions and to broader classes of f-divergences and integral constraint functions. The resulting theory substantially expands the scope of existing results and enables the construction of near-optimal statistical procedures—approaching theoretical lower bounds—for sequential testing, estimation, and change-point detection.

The original kernelized Kullback–Leibler (KL) divergence is ill-defined when the supports of compared distributions are disjoint—a fundamental limitation. To address this, the paper proposes a Tikhonov-regularized kernel KL divergence, constructed via covariance operator embeddings in a reproducing kernel Hilbert space (RKHS). This metric is well-defined for arbitrary probability distributions—including discrete, continuous, and mutually singular ones—and provides theoretical guarantees: a bias bound relative to the true KL divergence, finite-sample convergence rates, and a closed-form solution for discrete distributions. Furthermore, the authors formulate a Wasserstein gradient flow optimization framework for the proposed divergence, ensuring theoretical convergence, and design an efficient algorithm applicable to discrete structures such as point clouds. Experiments on point cloud transport tasks demonstrate that the method outperforms existing kernelized and Wasserstein-based approaches, achieving superior stability and robustness.

This work addresses the problem of learning probabilistic predictive models from supervisory signals given as possibility distributions in multiclass classification tasks. The authors propose transforming input possibility distributions into sets of probability distributions that satisfy compatibility and linear shape constraints, thereby introducing— for the first time—structured probabilistic set modeling for possibility-based supervision. Model outputs are efficiently calibrated to these sets via Kullback–Leibler projection, implemented through Dykstra’s algorithm combined with Bregman projections, ensuring minimal adjustment while preserving the ordinal relationships among class possibilities. Empirical evaluations on both synthetic data and the ChaosNLI natural language inference benchmark demonstrate the method’s effectiveness in achieving strong predictive performance and faithful order-preserving calibration.

Classical maximum entropy principle (MEP) relies on the system independence assumption in the Shore–Johnson axioms, which frequently fails in strongly correlated systems (e.g., economic or ecological networks), leading to systematic biases in Shannon-entropy-based inference. Method: We demonstrate that the Uffink–Jizba–Korbel (UJK) one-parameter generalized entropy family relaxes this assumption, offering a more robust entropy selection criterion for non-independent systems. By reformulating the Shore–Johnson axiomatization, we precisely delineate the domain of applicability for UJK entropies and establish a reproducible, transparent framework for entropy function selection and reporting. Contribution/Results: Empirical validation in economics (market interdependence modeling) and ecology (inference of species interactions) shows substantial improvements in distribution reconstruction accuracy and interpretability. This work is the first to systematically bridge foundational axiomatic principles with practical implementation guidelines, thereby advancing the reliable application of MEP in complex systems.

This study investigates whether different solvers systematically favor particular equilibria when multiple Nash equilibria exist in two-player zero-sum games. The authors construct benchmark games with analytically tractable Nash equilibrium sets and systematically evaluate the equilibrium selection behavior of algorithms including R-NaD, mirror descent, CFR, CFR+, and fictitious play. Their findings reveal that equilibrium selection is primarily governed by algorithmic type rather than random initialization: R-NaD consistently converges to the maximum-entropy equilibrium, whereas regret-minimization methods such as CFR+ tend to select low-entropy boundary equilibria. Further experiments demonstrate that the maximum-entropy equilibrium exhibits greater robustness against suboptimal opponents in Kuhn poker, highlighting the advantage of regularized last-iterate methods in achieving higher-quality equilibria.

This work addresses the difficulty traditional maximum entropy methods face in naturally handling bounded support and determining deformation parameters within unnormalized models. The authors propose a projective maximum entropy framework formulated in the projective space of non-negative measures, which unifies various entropy and divergence forms through linear moment constraints and establishes an exact correspondence between admissible regions and deformation parameters. By leveraging tools from projective geometry, q-exponential family distributions, and Mahalanobis-distance ellipsoidal constraints, they prove—for the first time—the universal equivalence of optimization solutions across different divergences. This yields a unique solution: a compactly supported q-Gaussian (or Student-type) density whose support exactly matches a prescribed ellipsoid, without requiring additional constraints, thereby providing a theoretically coherent reference distribution for robust estimation.

This study addresses the limitations of traditional maximum entropy methods, which are constrained to monomial moment constraints and thus struggle to accurately model non-Gaussian distributions exhibiting algebraic tails or symmetric multimodality, while also suffering from ill-conditioned dual problems. Building upon Kunchenko’s decomposition space, the work generalizes constraint basis functions to designable generators, introduces a tail-class-matching strategy for generator selection, and develops three novel generator families—fractional-power, trigonometric, and logarithmic-rational functions—supported by parity-admissibility theorems and a design atlas. Integrating a unified dual solver, an analytical product-moment evaluator, and a variance-optimal element selection rule (oPMM-alpha), the proposed framework reconstructs Student’s and Cauchy distributions under a single constraint and automatically selects fractional moments. Experiments demonstrate an 8.5-fold reduction in MSE for bimodal Gaussian mixtures; for heavy-tailed distributions, fractional-power generators achieve feasible recovery in 19 out of 20 trials with Kolmogorov–Smirnov statistic KS = 0.068, while logarithmic-rational generators precisely recover the Cauchy tail exponent using only one constraint.

This work addresses a theoretical gap in existing frameworks that combine distributional reinforcement learning with maximum-entropy control, which lack contraction guarantees for the distributional soft Bellman operator under the Cramér geometry. The authors construct a cumulative distribution function–based soft Bellman operator in the policy evaluation phase and, for the first time, prove its √γ-contraction property under the Cramér distance. They further demonstrate that its unique fixed point arises from a unified first-moment condition rather than conventional boundedness assumptions. Additionally, the paper establishes an equivalent spectral-domain representation of this operator in a Hilbert space. These results provide a rigorous theoretical foundation for designing critics and analyzing evaluation errors in distributional soft policy iteration algorithms.

This work addresses the challenges of model inaccuracy and ambiguous state distributions in nonlinear systems by proposing a distributionally robust optimization-based chance-constrained control framework. The approach constructs an ambiguity set using relative entropy constraints and derives an upper bound on risk expectations via the variational representation of the exponential integral. It further integrates nonlinear covariance propagation with adaptive determination of the ambiguity set radius based on second-order dynamic truncation error. Notably, the method recovers nominal risk in the zero-divergence limit, thereby overcoming the restrictive assumptions of Gaussianity prevalent in conventional approaches. Validation on spacecraft stochastic guidance tasks demonstrates that the proposed framework effectively enforces probabilistic safety constraints under distributional uncertainty.