time-varying model averaging

This study addresses the challenge of effectively integrating structural information from multiple channels in large-scale dynamic systems. To this end, the authors propose a time-varying multilayer network vector autoregressive (TV-MLN-VAR) model combined with a penalized model averaging approach that dynamically combines multiple candidate models to capture cross-channel spillover effects. The work innovatively introduces time-varying optimal model averaging weights and, for the first time, extends conformal prediction to locally stationary time series to construct valid prediction intervals. Theoretical analysis establishes the asymptotic optimality and convergence rate of the weight estimator. Extensive simulations and empirical applications to CPI inflation forecasting demonstrate that the proposed method achieves strong estimation and predictive performance even in finite samples.

In high-dimensional linear regression, where the number of parameters is comparable to the sample size, model-averaged predictions are susceptible to the double descent phenomenon and local risk divergence. This work reveals an emergent smoothing effect induced by weighted model aggregation near the interpolation threshold, which effectively mitigates risk divergence. Building on this insight, the authors propose Large Model Averaging (LaMA), a method that optimally balances bias and asymptotic variance to enhance both fitting accuracy and generalization performance. Theoretical analysis leverages random matrix theory and high-dimensional asymptotic risk characterizations, while empirical evaluations on both synthetic and real-world datasets demonstrate that LaMA significantly improves predictive accuracy, confirming its superiority over existing approaches.

Traditional exponential moving average (EMA) fails to suppress asymptotic noise in stochastic dynamical system trajectories due to its fixed decay rate, resulting in averaged estimates lacking strong stochastic convergence. To address this, we propose the *p-EMA* method, which introduces a subharmonic decaying weight sequence—assigning gradually diminishing yet persistently non-negligible weight to recent observations. Under mild weak autocorrelation assumptions, p-EMA establishes the first exponentially weighted averaging framework with rigorous strong convergence guarantees. We prove that p-EMA achieves almost sure convergence and $L^2$ convergence, with asymptotic variance vanishing to zero—overcoming fundamental limitations of standard EMA. Furthermore, integrating p-EMA into SGD gradient estimation enhances optimization stability. Empirical results demonstrate over 40% faster error convergence under non-stationary noise compared to conventional EMA-based estimators.

This paper addresses the challenges of inaccurate time-varying ensemble weight estimation and poor multi-step forecasting performance under model misspecification. We propose a Bayesian dynamic ensemble forecasting framework. Methodologically, it introduces a prediction prior driven by model diversity, coupled with a nonlinear state-space model and sequential Monte Carlo (particle filter) inference to enable real-time, adaptive updating of ensemble weights. The proposed prior automatically detects model redundancy, amplifies contributions from informative models, and provides diagnostic capabilities for model incompleteness and predictive uncertainty. Empirical evaluations on oil price forecasting and joint U.S. inflation–GDP forecasting demonstrate that our approach significantly outperforms equal-weight averaging, Bayesian model averaging, and conventional time-varying weighting methods. These results validate its robustness and superiority in complex, nonstationary environments.

In reinforcement learning, adaptive interaction data—where the behavior policy is nonstationary—invalidates standard estimators, undermining asymptotic normality for off-policy counterfactual policy evaluation and dynamic treatment effect (DTE) inference. To address this, we propose a weighted Z-estimation framework that constructs time-varying adaptive weights to stabilize heteroskedasticity, achieving, for the first time in the RL off-policy setting, both consistent and asymptotically normal DTE estimation. Our approach integrates dynamic causal inference with asymptotic statistical theory, enabling rigorous hypothesis testing and construction of uniformly valid confidence regions. Simulation studies and real-world RL experiments demonstrate substantial improvements in confidence interval coverage and statistical power. The method provides the first solution for structural parameter inference under adaptive experimentation that simultaneously offers theoretical guarantees—namely consistency, asymptotic normality, and uniform validity—and empirical robustness.

This work addresses the challenge of parameter estimation in dynamic models where the likelihood function is intractable, and existing likelihood-free methods either rely on handcrafted summary statistics or computationally expensive neural networks. To overcome these limitations, the authors propose a simulation-based inference approach leveraging random features. Their key innovation lies in introducing embedding theory from nonlinear dynamical systems into simulation-based inference, enabling identification of a p-dimensional parameter model by matching only a small number (2p+1) of random features between observed and simulated data. The method applies to both stationary and non-stationary processes and, under mild regularity conditions, yields consistent estimators. This framework establishes a new paradigm for dynamic system parameter estimation that is efficient, broadly applicable, and theoretically grounded.

This study addresses the limitations of existing model averaging approaches in handling spatially heterogeneous data, which are often sensitive to model misspecification and lack predictive flexibility. The authors propose the first extension of model averaging to the spatially varying coefficient framework, constructing a weighted average estimator from a set of candidate spatially varying coefficient models. The weights are determined dynamically via a Mallows-type criterion. Theoretical analysis demonstrates that the proposed method enjoys favorable asymptotic properties under both global misspecification and the presence of a quasi-correctly specified model. Extensive simulations and empirical applications confirm that the approach substantially outperforms existing methods in terms of prediction accuracy and robustness.

This work addresses the challenge of combining probabilistic predictions from multiple models in few-shot settings by proposing a general weighted averaging method grounded in a minimum divergence framework. Applicable to models constructed via frequentist, Bayesian, or other fitting paradigms, the approach employs a dual-motivation weighting scheme that simultaneously minimizes the divergence between the aggregated predictive distribution and the true data-generating distribution while accounting for model complexity. Theoretical analysis elucidates the source of its advantage under limited data regimes, and empirical evaluations demonstrate that the method consistently matches or significantly outperforms conventional model averaging strategies—such as Akaike weights and stacking—in terms of predictive accuracy, exhibiting robust performance across diverse scenarios.

This work addresses the significant performance variation of pretrained models on new tasks, where the relative superiority of models changes dynamically with input, making it challenging to select a single best model universally. To tackle this issue, the authors propose a localized model averaging approach that models fusion weights as functions of covariates and learns context-aware, dynamic weights within a general loss framework to adaptively combine the strengths of multiple models. This method overcomes the limitations of traditional static averaging and is theoretically shown to achieve asymptotic optimality in both in-sample and out-of-sample risk, along with consistent weight estimation. Extensive experiments demonstrate the effectiveness and robustness of the proposed approach across diverse prediction tasks.

This work addresses the lack of theoretical foundations in purely data-driven approaches for smoothing and forecasting in dynamical systems. It establishes, for the first time, a universal approximation theorem tailored to such tasks, rigorously proving the existence and approximability of the underlying operator mappings learned from data. By introducing a continuous-time neural operator architecture and integrating analyses of mapping existence with investigations into operator model properties, the study constructs a comprehensive theoretical framework. Experimental validation on canonical dynamical systems—including Lorenz '63, Lorenz '96, and Kuramoto–Sivashinsky—demonstrates the efficacy of the theoretical results, thereby providing a solid theoretical foundation for data-driven modeling of dynamical systems.