🤖 AI Summary
This paper addresses the equivalence between two definitions of weighted vector information—based on variability and non-uniformity—and establishes that Bregman divergence is the unique class of divergences ensuring their strict identity. Using tools from convex analysis, information geometry, and functional equations, we provide the first rigorous characterization showing that information-equivalence fully determines the functional form of the divergence. This yields a necessary and sufficient condition linking information-theoretic equivalence to divergence structure, thereby proving the uniqueness and foundational role of Bregman divergences within divergence theory. As a key contribution, we derive a novel axiomatic characterization of Bregman divergences, offering an essential theoretical criterion for divergence selection in optimization, statistics, and information theory.
📝 Abstract
Bregman divergences are a class of distance-like comparison functions which play fundamental roles in optimization, statistics, and information theory. One important property of Bregman divergences is that they cause two useful formulations of information content (in the sense of variability or non-uniformity) in a weighted collection of vectors to agree. In this note, we show that this agreement in fact characterizes the class of Bregman divergences; they are the only divergences which generate this agreement for arbitrary collections of weighted vectors.