Learning Linear Systems under Heavy-Tailed Noise: A Non-Asymptotic Analysis from A Single Trajectory

2026-09-30

AI Summary

This study addresses the unresolved non-asymptotic sample complexity of single-trajectory least squares estimation for linear systems under heavy-tailed noise. For exponentially stable systems, this work proposes a unified analytical framework that accommodates both sub-Gaussian and sub-exponential noise regimes, subject to persistence of excitation, bounded noise covariance, and finite moment conditions. The primary contribution lies in establishing a non-asymptotic error bound of $\widetilde{O}(r^{1/2}T^{-1/2+1/p})$, which is shown to be independent of the model order. This result significantly improves system identification performance in the presence of heavy-tailed noise, offering tighter theoretical guarantees than existing approaches.

Abstract

We establish non-asymptotic sample complexity bounds for the least-squares estimation of vector autoregressive models for exponentially stable systems with heavy-tailed noise based on a single observed trajectory. By assuming i.i.d. noise, bounded noise covariance, and persistent excitation, we show that the estimation error is $\widetilde{\mathcal{O}}(r^{1/2}T^{-1/2+1/p})$ under bounded $p$th moment for $p > 2$, where $T$ is the number of samples, $r$ is the noise dimension, and $\widetilde{\mathcal{O}}(\cdot)$ hides logarithmic terms. We also introduce a unifying approach to sample complexity analysis applicable to broad classes of noise distributions and showcase this by deriving error bounds for sub-exponential and sub-Gaussian noise distributions. Finally, we specialize our analysis to autoregressive models with exogenous inputs and show that the dimension factor of the error bound is independent of the model order.

Problem

• heavy-tailed noise
• linear systems
• vector autoregressive models
• non-asymptotic analysis
• sample complexity

Innovation

• heavy-tailed noise
• non-asymptotic analysis
• vector autoregressive models
• sample complexity
• single trajectory