New Lower Bound and Upper Bounds on the Regret for Online Sparse Linear Regression

2026-10-08

AI Summary

This study addresses the long-standing open problem of characterizing the information-theoretic complexity of online sparse linear regression. By integrating techniques from online learning, information-theoretic analysis, and sparse regression algorithms, this work establishes, for the first time, the minimax regret lower bound for this problem. Furthermore, it develops efficient algorithms that operate without regularity assumptions, thereby overcoming the strong dependence on model conditions inherent in traditional theoretical analyses. Beyond deriving tighter regret upper bounds, this research precisely characterizes the scaling relationships among key parameters, fully delineating the theoretical complexity boundaries of online sparse linear regression. Collectively, these contributions provide a foundational theoretical framework that advances the understanding of this domain.

Abstract

We study online sparse linear regression (OSLR) where any algorithm is restricted to accessing only $b$ out of $d$ attributes per instance for prediction and $b_0\geq 0$ additional attributes after prediction, which was proved to be NP-hard. Previous work focused on designing computationally efficient algorithms under regularity assumptions, but did not characterize its information theoretic complexity. In this work, we give the first lower bound on the minimax regret of OSLR and design algorithms with better upper bounds without regularity assumptions. We characterize how minimax regret scales with problem-dependent parameters, capturing the information theoretic complexity of OSLR.

Problem

• Online Sparse Linear Regression
• Minimax Regret
• Information Theoretic Complexity
• Lower Bound
• Upper Bounds

Innovation

• Online Sparse Linear Regression
• Minimax Regret
• Lower Bound
• Upper Bounds
• Information Theoretic Complexity