A complete characterisation of conditional entropies

2026-01-30

AI Summary

This work addresses the absence of a complete characterization of conditional entropy satisfying natural operational axioms. By introducing a set of operational principles—including additivity for independent random variables, invariance under relabeling, and monotonicity under conditional mixing channels—the authors provide the first systematic characterization of the most general form of conditional entropy. This family of entropies is shown to be expressible as an exponential average of Rényi entropies, parameterized by probability measures over the positive reals. The study further establishes a connection between this entropy family and the second law of quantum thermodynamics, demonstrating that these conditional entropies precisely determine the asymptotic rates of state transformations in the presence of side information, thereby yielding a new family of second laws for quantum thermodynamics.

Technology Category

• Machine Learning: Information Theory
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Security and Privacy: Applications of cryptography
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Entropies are fundamental measures of uncertainty with central importance in information theory and statistics and applications across all the quantitative sciences. Under a natural set of operational axioms, the most general form of entropy is captured by the family of R\'enyi entropies, parameterized by a real number $\alpha$. Conditional entropy extends the notion of entropy by quantifying uncertainty from the viewpoint of an observer with access to potentially correlated side information. However, despite their significance and the emergence of various useful definitions, a complete characterization of measures of conditional entropy that satisfy a natural set of operational axioms has remained elusive. In this work, we provide a complete characterization of conditional entropy, defined through a set of axioms that are essential for any operationally meaningful definition: additivity for independent random variables, invariance under relabeling, and monotonicity under conditional mixing channels. We prove that the most general form of conditional entropy is captured by a family of measures that are exponential averages of R\'enyi entropies of the conditioned distribution and parameterized by a real parameter and a probability measure on the positive reals. Finally, we show that these quantities determine the rate of transformation under conditional mixing and provide a set of second laws of quantum thermodynamics with side information for states diagonal in the energy eigenbasis.

Problem

• conditional entropy
• Rényi entropy
• operational axioms
• information theory
• uncertainty quantification

Innovation

• conditional entropy
• Rényi entropy
• operational axioms
• quantum thermodynamics
• side information