Pinsker's inequality for adapted total variation

2025-06-27

AI Summary

This work addresses the problem of quantifying distributional divergence between stochastic processes, specifically investigating whether a Pinsker-type inequality exists between the adapted total variation (ATV) distance and the relative entropy (KL divergence) $H(mu| u)$. For discrete-time stochastic processes, we derive and rigorously prove a tight upper bound: $mathrm{ATV}(mu, u) leq sqrt{n}sqrt{2H(mu| u)}$, where $n$ denotes the process horizon. Methodologically, our approach integrates an adaptive extension of the Wasserstein distance, discrete measure-theoretic techniques, and refined information-theoretic inequalities. This result constitutes the first incorporation of temporal adaptivity into the classical Pinsker framework, overcoming the traditional reliance of total variation on non-adaptive settings. The bound is provably tight and significantly enhances both precision and applicability in comparing stochastic processes. It provides a novel theoretical tool for distributional sensitivity analysis in stochastic control, online learning, and sequential modeling.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Calibration & Uncertainty Quantification
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Pinsker's classical inequality asserts that the total variation $TV(μ, ν)$ between two probability measures is bounded by $sqrt{ 2H(μ|ν)}$ where $H$ denotes the relative entropy (or Kullback-Leibler divergence). Considering the discrete metric, $TV$ can be seen as a Wasserstein distance and as such possesses an adapted variant $ATV$. Adapted Wasserstein distances have distinct advantages over their classical counterparts when $μ, ν$ are the laws of stochastic processes $(X_k)_{k=1}^n, (Y_k)_{k=1}^n$ and exhibit numerous applications from stochastic control to machine learning. In this note we observe that the adapted total variation distance $ATV$ satisfies the Pinsker-type inequality $$ ATV(μ, ν)leq sqrt{n} sqrt{2 H(μ|ν)}.$$

Problem

• Extends Pinsker's inequality to adapted total variation distance
• Compares laws of stochastic processes using ATV and entropy
• Derives bound for ATV in terms of relative entropy

Innovation

• Adapted total variation for stochastic processes
• Pinsker-type inequality with adapted Wasserstein distance
• Bounding ATV by relative entropy and n