Rate-Distortion-Classification Representation Theory for Bernoulli Sources

2026-01-17

AI Summary

This study addresses lossy compression for binary classification tasks, aiming to optimize the trade-off between rate-distortion and classification performance. Within the one-shot public randomness framework, and considering a Bernoulli source, Hamming distortion, and a binary symmetric classifier, the work presents the first closed-form characterizations of the achievable regions for rate-distortion-classification (RDC) and distortion-classification (DC). It further introduces a linear programming–based method to characterize the lower boundary of the DC region. The authors also derive computable upper and lower bounds on the minimal asymptotic rate of universal encoders supporting multiple operating points, thereby fully characterizing the theoretical limits of RDC and DC. Numerical experiments validate the proposed theory, offering tight rate-loss bounds and practical design guidelines for task-oriented compression.

Technology Category

• Machine Learning: Information Theory
• Data Mining & Knowledge Management: Data Compression
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation

Abstract

We study task-oriented lossy compression through the lens of rate-distortion-classification (RDC) representations. The source is Bernoulli, the distortion measure is Hamming, and the binary classification variable is coupled to the source via a binary symmetric model. Building on the one-shot common-randomness formulation, we first derive closed-form characterizations of the one-shot RDC and the dual distortion-rate-classification (DRC) tradeoffs. We then use a representation-based viewpoint and characterize the achievable distortion-classification (DC) region induced by a fixed representation by deriving its lower boundary via a linear program. Finally, we study universal encoders that must support a family of DC operating points and derive computable lower and upper bounds on the minimum asymptotic rate required for universality, thereby yielding bounds on the corresponding rate penalty. Numerical examples are provided to illustrate the achievable regions and the resulting universal RDC/DRC curves.

Problem

• rate-distortion-classification
• task-oriented compression
• Bernoulli source
• universal encoder
• distortion-rate-classification

Innovation

• rate-distortion-classification
• one-shot representation
• universal encoder
• distortion-classification tradeoff
• Bernoulli source