An information identity reveals the geometry of deviation events

2026-09-29

AI Summary

This study investigates the geometric decomposition mechanism of probability exponents in empirical measure deviation events. By integrating large deviation theory, information theory, and group action analysis, it proposes a novel information identity that precisely decomposes the probability exponent into a relative entropy rate and a conditional dependence term. For the first time, this work quantifies the competitive relationship between these two components and their critical transition points within large deviations from a geometric perspective, revealing how they evolve with the underlying set geometry and providing exact solutions for exponentially transformed expectations. Furthermore, it elucidates the asymptotic behavior of the dependence proportion under varying geometric structures, thereby establishing a new theoretical framework for large deviation analysis.

Abstract

In a deviation event, the empirical measure of $n$ independent draws lands in a set of distributions. The exponent of its probability splits, at every $n$, into two nonnegative terms. The first is the rate: $n$ times the relative entropy, from the population, of the distribution of a typical draw under the event. The second is the dependence that conditioning induces among the draws, measured by their total correlation. Their relative sizes depend on the geometry of the set. This paper focuses on this exact split and on the geometry that the interplay of its two terms reveals. The dependence vanishes exactly when the event confines every draw to a single set, and the classical rate of that constraint is exact. On a set invariant under a compact group that fixes the population and leaves no invariant set of intermediate probability, the rate vanishes; uniform convergence over a hypothesis class that such a group permutes is one. On a half-space whose threshold sits a fixed number of standard errors above the mean, the fraction of the exponent that is dependence tends to a function of the event's probability alone. On a set split into pieces, the dependence is the pieces' average plus $n$ times the Jensen-Shannon divergence of their marginals, minus the entropy of their weights. The split extends to a general reference law. When the information projection is such a law, the split gives the exact value of the expectation in the exponential change of measure to the projection.

Problem

• deviation event
• information identity
• relative entropy
• total correlation
• probability exponent

Innovation

• Information Identity
• Deviation Events
• Total Correlation
• Relative Entropy
• Geometric Structure