Out-of-Distribution generalization of quantile regression with heavy tailed inputs: an SVM approach

2026-05-29

AI Summary

This study addresses the challenge of poor generalization in quantile regression under covariate distributional shift, particularly when extreme heavy-tailed covariate values are present. By integrating multivariate extreme value theory with statistical learning under the assumption of regular variation, the authors develop an asymptotic conditional risk minimization framework that models the angular component of extreme observations. They propose, for the first time, a reproducing kernel Hilbert space support vector machine approach for extreme quantile regression that requires no variable transformation and accommodates unbounded response variables, accompanied by finite-sample theoretical guarantees. Empirical validation on Danube River runoff data demonstrates the method’s strong extrapolation capability in the covariate tails and confirms its theoretical validity.

Abstract

We study quantile regression in an extrapolation regime where the covariate takes unusually large values. Under regular variation assumptions, extreme observations can be effectively characterized through their angular components, enabling learning strategies that focus on the angle of the most extreme observations. This approach is formalized through the minimization of an asymptotic conditional risk that localizes learning in the tail of the covariate distribution. We propose a novel Support Vector Machine (SVM) framework for extreme quantile regression, leveraging reproducing kernel Hilbert spaces to handle high-dimensional and nonlinear settings. Our method also accommodates unbounded response variables and avoids restrictive transformations. We establish finite-sample learning guarantees under mild regularity assumptions. The proposed framework unifies ideas from statistical learning and multivariate extremes, providing a tractable and theoretically grounded approach to extrapolation. We complement our theoretical findings with an empirical study on river flow data from the Danube, demonstrating the practical relevance of our methods.

Problem

• Out-of-Distribution generalization
• quantile regression
• heavy-tailed inputs
• extreme value extrapolation
• tail behavior

Innovation

• extreme quantile regression
• out-of-distribution generalization
• support vector machine
• regular variation
• reproducing kernel Hilbert space