Shortcut to Nowhere: Demystifying Deep Spurious Regression

2026-06-01

AI Summary

This work addresses the challenge of model generalization failure in continuous prediction tasks caused by spurious correlations between attributes and target variables. It formally defines and systematically investigates, for the first time, the problem of Deep Spurious Regression (DSR). The authors propose a novel approach that jointly models attribute similarity in both label and feature spaces, leveraging a distribution calibration mechanism to align feature and label distributions across different attributes. This alignment enhances model generalization to unseen attribute–target combinations. Key contributions include the construction of the first multi-domain DSR benchmark dataset and extensive experiments across real-world scenarios—spanning computer vision, environmental sensing, and regression with large language models—demonstrating significant performance gains over existing methods and validating the effectiveness of the proposed debiasing strategy.

Abstract

Real-world regression often exhibits shortcuts: attributes that are spuriously correlated with continuous targets in training, yet unreliable under deployment shifts; regressing targets using such shortcuts may fail catastrophically at test time. Existing studies on spurious correlations focus primarily on classification, where labels are categorical and groups are naturally defined. However, many real-world tasks require continuous prediction, where hard label boundaries or discrete group-label pairs do not exist. We define Deep Spurious Regression (DSR) as learning from regression data with attribute-label confounding, addressing continuous spurious correlations, and generalizing to all attribute-label combinations at test time. Motivated by the intrinsic difference between classification and regression shortcuts, we propose to exploit the similarity among spurious attributes in both label and feature spaces, thereby accounting for nearby targets and related groups while calibrating both label and learned feature distributions across attributes. Extensive experiments on common real-world DSR datasets that span computer vision, environmental sensing, and large language model (LLM) regression verify the superior performance of our strategies. Our work fills the gap in benchmarks and techniques for studying spurious correlations in continuous prediction.

Problem

• spurious correlation
• regression
• continuous prediction
• attribute-label confounding
• distribution shift

Innovation

• Deep Spurious Regression
• spurious correlation
• continuous prediction
• distribution calibration
• attribute-label confounding