On Distributionally Robust Lossy Source Coding

2025-07-23

AI Summary

This paper investigates distributionally robust lossy source coding under source distribution uncertainty, addressing both single-shot and asymptotic regimes. For an uncertainty set defined by a Kullback–Leibler divergence ball, we extend the strong functional representation lemma to families of distributions for the first time and propose a joint coupling optimization framework, yielding an implicit characterization of the robust rate-distortion function. We further derive an analytical expression for the Bernoulli source and design an efficient numerical algorithm for its computation. This work establishes the first rate-distortion theory framework tailored to distributional robustness, overcoming the classical assumption of perfect knowledge of the true source distribution. It provides a novel paradigm and practical computational tools for coding design under distributional uncertainty.

Technology Category

• Reasoning under Uncertainty: Uncertainty Representations
• Intelligent Robots: Learning & Optimization for ROB
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

In this paper, we investigate the problem of distributionally robust source coding, i.e., source coding under uncertainty in the source distribution, discussing both the coding and computational aspects of the problem. We propose two extensions of the so-called Strong Functional Representation Lemma (SFRL), considering the cases where, for a fixed conditional distribution, the marginal inducing the joint coupling belongs to either a finite set of distributions or a Kullback-Leibler divergence sphere (KL-Sphere) centered at a fixed nominal distribution. Using these extensions, we derive distributionally robust coding schemes for both the one-shot and asymptotic regimes, generalizing previous results in the literature. Focusing on the case where the source distribution belongs to a given KL-Sphere, we derive an implicit characterization of the points attaining the robust rate-distortion function (R-RDF), which we later exploit to implement a novel algorithm for computing the R-RDF. Finally, we characterize the analytical expression of the R-RDF for Bernoulli sources, providing a theoretical benchmark to evaluate the estimation performance of the proposed algorithm.

Problem

• Study distributionally robust source coding under uncertainty
• Extend Strong Functional Representation Lemma for robust coding
• Develop algorithm to compute robust rate-distortion function

Innovation

• Extends Strong Functional Representation Lemma
• Derives robust coding schemes
• Implements novel R-RDF algorithm