Rate-distortion Theory with Lower Semi-continuous Distortion: A Concentration-compactness Approach

2026-01-12

AI Summary

This work addresses the challenge in classical rate-distortion theory where the existence of an optimal reconstruction distribution cannot be guaranteed due to the distortion function lacking continuity or compact support. For the first time, the concentration-compactness principle is introduced into the rate-distortion analysis framework. Under the mild assumptions that the distortion function is lower semicontinuous and satisfies a coercivity condition, the authors rigorously establish the existence of a minimizer for the rate-distortion functional by combining tools from functional analysis and information theory. This result overcomes the traditional reliance on continuity and compact support, yielding a more general and unified existence theorem that significantly broadens the applicability of rate-distortion theory to scenarios involving discontinuous distortion measures.

Technology Category

• Machine Learning: Information Theory
• Constraint Satisfaction and Optimization: Other Foundations of Constraint Satisfaction
• Search and Optimization: Non-convex Optimization

Application Category

• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Responsible Web: Measurement, analysis, and circumvention of Web censorship

Abstract

In this paper, we study rate-distortion theory for general sources with an emphasis on the existence of optimal reconstruction distributions. Classical existence results rely on compactness assumptions with continuous distortion that are often violated in general settings. By introducing the concentration-compactness principle into the analysis of the rate-distortion functional, we establish the existence of optimal reconstructions under mild coercivity and lower semi-continuity conditions on the distortion function. Our results provide a unified and transparent existence theorem for rate-distortion problems with lower semi-continuous distortion.

Problem

• rate-distortion theory
• optimal reconstruction
• lower semi-continuous distortion
• existence
• general sources

Innovation

• concentration-compactness
• rate-distortion theory
• lower semi-continuous distortion
• optimal reconstruction
• existence theorem