Variational meta-learning inference for low dimensional neural system identification

2026-07-21

AI Summary

This work addresses the challenges of overfitting and poorly calibrated uncertainty quantification in neural system identification under low-data regimes by proposing a fully probabilistic manifold meta-learning framework based on amortized variational inference. The approach introduces probabilistic modeling into manifold meta-learning for the first time, learning a generative prior over a low-dimensional parameter manifold and combining maximum a posteriori estimation with Laplace approximation to yield task-adaptive posterior approximations. Experimental results demonstrate that the method achieves prediction accuracy comparable to deterministic approaches on both static regression and Bouc–Wen dynamic system benchmarks, while providing well-calibrated uncertainty estimates even under extremely limited data conditions.

Abstract

Deep learning has proven highly effective for nonlinear system identification, but heavily parameterized neural networks are prone to overfitting in low-data regimes and lack reliable uncertainty quantification. The recently developed manifold meta-learning framework addresses the data efficiency problem by restricting the model parameters to a meta-learned low-dimensional manifold. However, that method is purely deterministic. We propose a fully probabilistic extension of the manifold meta-learning framework, based on amortized Variational Inference, where a generative prior over the low-dimensional parameter manifold is learned. During task-specific adaptation, we combine Maximum A Posteriori estimation with the Laplace approximation to yield a mathematically grounded posterior approximation. Evaluated on a static regression task and the Bouc--Wen dynamical system benchmark, the proposed approach achieves predictive accuracy comparable to its deterministic counterpart while successfully providing calibrated uncertainty bounds in severely low-data regimes.

Problem

• system identification
• low-data regime
• uncertainty quantification
• overfitting
• neural networks

Innovation

• variational inference
• manifold meta-learning
• uncertainty quantification
• low-data regime
• neural system identification