The Complexity of Sequential Prediction in Dynamical Systems

2024-02-09

arXiv.org

AI Summary

This paper addresses the problem of next-state prediction for model-free dynamical systems—i.e., predicting future states without knowledge of the underlying evolution function and without parametric assumptions. Methodologically, it introduces novel combinatorial metrics and dimensions to characterize the fundamental statistical complexity of model-free time-series prediction for the first time. Leveraging tools from combinatorial learning theory, dynamical systems analysis, and online learning regret analysis, the paper rigorously derives tight optimal mistake bounds (in the realizable setting) and regret bounds (in the agnostic setting). The results establish the first systematic theoretical foundation for model-free time-series prediction, precisely quantifying its intrinsic statistical difficulty and delineating fundamental algorithmic limits.

Technology Category

Machine Learning: Other Foundations of Machine LearningMultiagent Systems: Other Foundations of Multi Agent SystemsSearch and Optimization: Learning to Search

Application Category

Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphsEconomics, Online Markets and Human Computation: Cost models of using LLMs in production systemsUser Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

We study the problem of learning to predict the next state of a dynamical system when the underlying evolution function is unknown. Unlike previous work, we place no parametric assumptions on the dynamical system, and study the problem from a learning theory perspective. We define new combinatorial measures and dimensions and show that they quantify the optimal mistake and regret bounds in the realizable and agnostic setting respectively.

Problem

• Predicting next state in unknown dynamical systems without parametric assumptions
• Quantifying optimal mistake and regret bounds via new combinatorial measures
• Characterizing mistake growth and regret rates in realizable versus agnostic settings

Innovation

• No parametric assumptions on dynamical systems
• New combinatorial measures quantify bounds
• Mistakes grow with any increasing function