Directional Confusions Reveal Divergent Inductive Biases Through Rate-Distortion Geometry in Human and Machine Vision

2026-04-23

AI Summary

This work reveals that despite comparable classification accuracy, humans and deep visual models exhibit systematically distinct patterns of directed confusion, reflecting fundamental differences in their inductive biases. The study introduces the first integration of directed confusion analysis with rate–distortion geometry, proposing three interpretable geometric descriptors—slope (β), curvature (κ), and efficiency (AUC)—to quantify the strength and breadth of confusion. Findings show that human confusion is characterized by broad yet weak asymmetry, whereas models display sparse but intense directional collapse. Notably, even robustly trained models fail to replicate the similarity structure inherent in human perception. This approach offers a novel perspective and quantitative framework for dissecting perceptual discrepancies between biological and artificial vision systems.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Humans and AI: Other Foundations of Human Computation & AI

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Humans and modern vision models can reach similar classification accuracy while making systematically different kinds of mistakes - differing not in how often they err, but in who gets mistaken for whom, and in which direction. We show that these directional confusions reveal distinct inductive biases that are invisible to accuracy alone. Using matched human and deep vision model responses on a natural-image categorization task under 12 perturbation types, we quantify asymmetry in confusion matrices and link it to generalization geometry through a Rate-Distortion (RD) framework, summarized by three geometric signatures (slope (beta), curvature (kappa)) and efficiency (AUC). We find that humans exhibit broad but weak asymmetries, whereas deep vision models show sparser, stronger directional collapses. Robustness training reduces global asymmetry but fails to recover the human-like breadth-strength profile of graded similarity. Mechanistic simulations further show that different asymmetry organizations shift the RD frontier in opposite directions, even when matched for performance. Together, these results position directional confusions and RD geometry as compact, interpretable signatures of inductive bias under distribution shift.

Problem

• directional confusions
• inductive biases
• rate-distortion geometry
• asymmetry
• distribution shift

Innovation

• directional confusions
• inductive biases
• Rate-Distortion geometry
• asymmetry in confusion matrices
• generalization geometry