Fast and Interpretable Autoregressive Estimation with Neural Network Backpropagation

2026-03-19

AI Summary

This work addresses the high computational cost and frequent convergence failures associated with parameter estimation in traditional autoregressive models. The authors propose embedding the autoregressive structure within a feedforward neural network, enabling coefficient estimation via backpropagation and gradient descent for the first time. This approach preserves model interpretability while substantially improving computational efficiency and numerical stability. Experimental results on 125,000 synthetic time series demonstrate that the method achieves a 100% success rate in recovering true coefficients, markedly outperforming conditional maximum likelihood estimation—which fails in 55% of cases—and delivers up to a 34.2-fold speedup.

Technology Category

Machine Learning: Deep Generative Models & AutoencodersReasoning under Uncertainty: Relational Probabilistic ModelsCognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphsUser Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalizationEconomics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Autoregressive (AR) models remain widely used in time series analysis due to their interpretability, but convencional parameter estimation methods can be computationally expensive and prone to convergence issues. This paper proposes a Neural Network (NN) formulation of AR estimation by embedding the autoregressive structure directly into a feedforward NN, enabling coefficient estimation through backpropagation while preserving interpretability. Simulation experiments on 125,000 synthetic AR(p) time series with short-term dependence (1 <= p <= 5) show that the proposed NN-based method consistently recovers model coefficients for all series, while Conditional Maximum Likelihood (CML) fails to converge in approximately 55% of cases. When both methods converge, estimation accuracy is comparable with negligible differences in relative error, R2 and, perplexity/likelihood. However, when CML fails, the NN-based approach still provides reliable estimates. In all cases, the NN estimator achieves substantial computational gains, reaching a median speedup of 12.6x and up to 34.2x for higher model orders. Overall, results demonstrate that gradient-descent NN optimization can provide a fast and efficient alternative for interpretable AR parameter estimation.

Problem

• Autoregressive models
• parameter estimation
• convergence issues
• computational efficiency
• time series analysis

Innovation

• autoregressive estimation
• neural network backpropagation
• interpretable time series modeling
• computational efficiency
• gradient-based optimization