Glivenko-Cantelli for $f$-divergence

2025-03-21

AI Summary

将Glivenko-Cantelli定理从总变差距离推广到所有f-散度，关键贡献是解决了在π系统（非σ子代数）上定义f-散度的问题，并保持了f-散度的性质。

Technology Category

• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Machine Learning: Information Theory
• Constraint Satisfaction and Optimization: Other Foundations of Constraint Satisfaction

Application Category

• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Responsible Web: Measurement, analysis, and circumvention of Web censorship

Abstract

We extend the celebrated Glivenko-Cantelli theorem, sometimes called the fundamental theorem of statistics, from its standard setting of total variation distance to all $f$-divergences. A key obstacle in this endeavor is to define $f$-divergence on a subcollection of a $sigma$-algebra that forms a $pi$-system but not a $sigma$-subalgebra. This is a side contribution of our work. We will show that this notion of $f$-divergence on the $pi$-system of rays preserves nearly all known properties of standard $f$-divergence, yields a novel integral representation of the Kolmogorov-Smirnov distance, and has a Glivenko-Cantelli theorem.

Problem

• Extend Glivenko-Cantelli theorem to f-divergences
• Define f-divergence on π-systems, not σ-subalgebras
• Preserve properties of f-divergence on ray π-systems

Innovation

• Extends Glivenko-Cantelli theorem to f-divergences
• Defines f-divergence on π-systems, not σ-subalgebras
• Novel integral representation for Kolmogorov-Smirnov distance