A Simple and Effective Random Forest Modelling for Nonlinear Time Series Data

2025-11-09

AI Summary

To address the limitation of standard Random Forests (RF) in capturing temporal dependencies for nonlinear time series forecasting, this paper proposes RF-RW—a novel RF variant that preserves sequential structure. Unlike conventional RF, RF-RW abandons bootstrap resampling to maintain temporal integrity and introduces a random weighting mechanism to reduce inter-tree correlation. Theoretically, we establish the first non-asymptotic concentration bound and uniform convergence guarantee for this framework, applicable to both high-dimensional and fixed-dimensional feature spaces. Empirically, RF-RW achieves statistically significant improvements over baseline methods—including standard RF, SVM, and LSTM—on synthetic time series and real-world forecasting of daily new COVID-19 cases in the UK. These results demonstrate RF-RW’s superior capability in modeling temporal dynamics and generalizing to unseen sequences.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Reasoning under Uncertainty: Sequential Decision Making
• Planning, Routing, and Scheduling: Temporal Planning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

In this paper, we propose Random Forests by Random Weights (RF-RW), a theoretically grounded and practically effective alternative RF modelling for nonlinear time series data, where existing RF-based approaches struggle to adequately capture temporal dependence. RF-RW reconciles the strengths of classic RF with the temporal dependence inherent in time series forecasting. Specifically, it avoids the bootstrap resampling procedure, therefore preserves the serial dependence structure, whilst incorporates independent random weights to reduce correlations among trees. We establish non-asymptotic concentration bounds and asymptotic uniform consistency guarantees, for both fixed- and high-dimensional feature spaces, which extend beyond existing theoretical analyses of RF. Extensive simulation studies demonstrate that RF-RW outperforms existing RF-based approaches and other benchmarks such as SVM and LSTM. It also achieves the lowest error among competitors in our real-data example of predicting UK COVID-19 daily cases.

Problem

• Modeling nonlinear time series with temporal dependence
• Improving Random Forest performance for sequential data
• Overcoming limitations of existing tree-based forecasting methods

Innovation

• Random Forests with random weights for time series
• Avoids bootstrap to preserve serial dependence structure
• Uses independent random weights to reduce tree correlations