Rate-Distortion Signatures of Generalization and Information Trade-offs

2026-03-02

AI Summary

Existing robustness metrics struggle to characterize the trade-off between accuracy and robustness in visual systems. This work introduces rate-distortion theory for the first time to model stimulus-response mappings as a communication channel, extracting the rate-distortion frontier from confusion matrices and quantifying the trade-off via two geometric features: slope (β) and curvature (κ). The approach is model-agnostic, interpretable, and enables direct comparison of generalization behaviors between humans and deep neural networks. Experiments reveal that both humans and models adhere to lossy compression principles, yet humans exhibit a smoother and more flexible trade-off, whereas deep networks display steeper, more brittle frontiers—even at matched accuracy levels. Although robust training improves performance, it does not necessarily align models’ generalization strategies with those of humans.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Generalization to novel visual conditions remains a central challenge for both human and machine vision, yet standard robustness metrics offer limited insight into how systems trade accuracy for robustness. We introduce a rate-distortion-theoretic framework that treats stimulus-response behavior as an effective communication channel, derives rate-distortion (RD) frontiers from confusion matrices, and summarizes each system with two interpretable geometric signatures - slope ($β$) and curvature ($κ$) - which capture the marginal cost and abruptness of accuracy-robustness trade-offs. Applying this framework to human psychophysics and 18 deep vision models under controlled image perturbations, we compare generalization geometry across model architectures and training regimes. We find that both biological and artificial systems follow a common lossy-compression principle but occupy systematically different regions of RD space. In particular, humans exhibit smoother, more flexible trade-offs, whereas modern deep networks operate in steeper and more brittle regimes even at matched accuracy. Across training regimes, robustness training induces systematic but dissociable shifts in beta/kappa, revealing cases where improved robustness or accuracy does not translate into more human-like generalization geometry. These results demonstrate that RD geometry provides a compact, model-agnostic lens for comparing generalization behavior across systems beyond standard accuracy-based metrics.

Problem

• generalization
• robustness
• rate-distortion
• accuracy-robustness trade-off
• visual perception

Innovation

• rate-distortion theory
• generalization geometry
• robustness-accuracy trade-off
• confusion matrix
• model-agnostic evaluation