STLDM: Spatio-Temporal Latent Diffusion Model for Precipitation Nowcasting

2025-12-24

AI Summary

Precipitation nowcasting faces dual challenges: deterministic models suffer from inherent blur, while generative models exhibit limited accuracy. To address this, we propose a two-stage short-term forecasting framework. In the first stage, a spatiotemporal conditional network generates high-accuracy deterministic predictions. In the second stage, a latent diffusion model (LDM) is jointly optimized with a variational autoencoder (VAE) in latent space—enabling end-to-end stochastic modeling and, for the first time, co-training of LDM, VAE, and the conditional network. This design decouples deterministic forecasting from uncertainty quantification, balancing structural fidelity and sample diversity. Evaluated on multiple radar echo datasets, our method significantly outperforms state-of-the-art approaches in both forecast accuracy and fine-grained detail preservation, while reducing inference computational overhead.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Computer Vision: Diffusion Models for Vision
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Precipitation nowcasting is a critical spatio-temporal prediction task for society to prevent severe damage owing to extreme weather events. Despite the advances in this field, the complex and stochastic nature of this task still poses challenges to existing approaches. Specifically, deterministic models tend to produce blurry predictions while generative models often struggle with poor accuracy. In this paper, we present a simple yet effective model architecture termed STLDM, a diffusion-based model that learns the latent representation from end to end alongside both the Variational Autoencoder and the conditioning network. STLDM decomposes this task into two stages: a deterministic forecasting stage handled by the conditioning network, and an enhancement stage performed by the latent diffusion model. Experimental results on multiple radar datasets demonstrate that STLDM achieves superior performance compared to the state of the art, while also improving inference efficiency. The code is available in https://github.com/sqfoo/stldm_official.

Problem

• Develops a diffusion-based model for precipitation nowcasting
• Addresses blurry predictions in deterministic models
• Improves accuracy and efficiency in weather forecasting

Innovation

• Uses latent diffusion model for precipitation nowcasting
• Combines deterministic forecasting with enhancement stages
• Learns end-to-end latent representation with VAE