Regularising NARX models with multi-task learning

2025-01-08

AI Summary

NARX models suffer from overfitting and poor generalization in high-noise time-series forecasting. To address this, we propose the first multi-task learning framework specifically designed for NARX modeling: instead of performing single-step prediction alone, the model simultaneously predicts the current output and multiple future outputs. The long-horizon forecasting tasks act as implicit regularization constraints, enhancing robustness without introducing additional hyperparameters or architectural modifications. Crucially, temporal consistency is enforced solely through task coupling—embedding a natural inductive bias into the learning process. Evaluated on high-noise benchmark datasets, our approach achieves significant reductions in normalized mean squared error (NMSE) compared to standard single-task NARX baselines, empirically validating the efficacy of multi-task regularization. Our core contribution lies in the systematic integration of multi-task learning into NARX modeling—a novel paradigm that substantially improves generalization capability for nonlinear autoregressive time-series models.

Technology Category

• Machine Learning: Multimodal Learning
• Natural Language Processing: Learning & Optimization for NLP
• Multiagent Systems: Multiagent Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

A Nonlinear Auto-Regressive with eXogenous inputs (NARX) model can be used to describe time-varying processes; where the output depends on both previous outputs and current/previous external input variables. One limitation of NARX models is their propensity to overfit and result in poor generalisation for future predictions. The proposed method to help to overcome the issue of overfitting is a NARX model which predicts outputs at both the current time and several lead times into the future. This is a form of multi-task learner (MTL); whereby the lead time outputs will regularise the current time output. This work shows that for high noise level, MTL can be used to regularise NARX with a lower Normalised Mean Square Error (NMSE) compared to the NMSE of the independent learner counterpart.

Problem

• NARX Model
• Overfitting
• Time-series Data

Innovation

• Multi-task Learning
• NARX Model Improvement
• Noise Robustness