🤖 AI Summary
This work addresses the limitations of existing data-driven nowcasting methods, which rely on deterministic encoding to compress high-dimensional meteorological data and consequently struggle to model uncertainty during decoding—particularly under extreme weather conditions. To overcome this, the authors propose FREUD, a novel framework that, for the first time, integrates uncertainty-preserving frame-wise encoding with a unified video decoding architecture. Built upon rectified flow Transformers, FREUD efficiently captures spatiotemporal stochasticity within a compact latent space while enabling continuous forecast updates and temporally coherent generation. Evaluated on the SEVIR benchmark, the method achieves state-of-the-art performance, and its reliability in predicting extreme precipitation events is further enhanced through both model design and test-time scaling strategies.
📝 Abstract
Accurate weather forecasts are essential across various domains and are safety-critical in extreme weather conditions. Compared to simulation-based forecasting, data-driven approaches show greater efficiency, enabling short-term, high-resolution nowcasting. In particular, diffusion models proved effective in weather nowcasting due to their strong probabilistic foundation. However, existing methods rely on deterministic compression to reduce the complexity of high-dimensional weather data, limiting their ability to capture uncertainty in the decoding process. In this work, we introduce $\textbf{FREUD}$, a $\textbf{Fr}$ame-wise $\textbf{E}$ncoder and $\textbf{U}$nited $\textbf{D}$ecoder model based on rectified flow transformers for efficient compression of spatio-temporal weather data. Frame-wise encoding enables continuous forecast updates, while the unified video decoder ensures temporal consistency. Our uncertainty-preserving first stage allows us to capture aleatoric uncertainty via ensembling, which is particularly beneficial for extreme weather events with high decoding variability. We achieve state-of-the-art performance in precipitation nowcasting with a compact latent-space rectified flow transformer on the SEVIR benchmark and show further performance gains by model and test-time scaling. Code available here: https://github.com/CompVis/weather-rf