Problem
Research questions and friction points this paper is trying to address.
Data Assimilation
Score-based Models
Autoregressive Diffusion Models
Uncertainty
Dynamical Systems
EnJoi: Ensemble Joint Score Filter for Generative Data Assimilation
🤖 AI Summary
This study addresses the limitation of autoregressive diffusion models in data assimilation, where historical prediction uncertainty is often neglected. To overcome this, we propose a generative assimilation algorithm that integrates principles from the ensemble Kalman filter. Built upon score-based generative models and the four-dimensional variational framework (En4DVar), the method dynamically corrects uncertainty propagation through ensemble particle covariance estimation. By learning a joint state distribution encompassing both past and future states, it achieves an adaptive balance between current state confidence and incoming observations. Experimental evaluations on fluid dynamics and traffic flow simulation tasks demonstrate that the proposed algorithm significantly enhances state reconstruction performance under sparse and non-uniform observation conditions.
📝 Abstract
Data Assimilation (DA) aims to recover the full state of a dynamical system that is only partially observed. A solution is to use Score-based models to generate physically consistent trajectories that agree with the observations. These Autoregressive Diffusion models are trained by conditioning on the previous state; however, they do not take into account the uncertainty of their past predictions. We propose a new diffusion-based assimilation algorithm that dynamically balances the confidence in the current state and the new observations. Crucially, we choose to learn the distribution of the joint state containing both the past and future. This allows us to use a modified version of En4DVar, a classical DA algorithm that relies on the covariance of an ensemble of particles. Experiments on fluid and traffic flow simulations show improved reconstruction performance, especially in situations where observations are sparse and non-homogeneous.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Data Assimilation
Score-based Diffusion Models
Joint State Distribution
En4DVar
Generative Modeling
🔎 Similar Papers
No similar papers found.
