Data-driven Seasonal Climate Predictions via Variational Inference and Transformers

2025-03-26

AI Summary

Existing seasonal forecasting methods face a dual bottleneck: physics-based models (e.g., GCMs) incur prohibitive computational costs, while statistical and machine learning approaches suffer from limited historical records or inadequate spatiotemporal generalization. This paper proposes the first generative framework integrating variational inference with a spatiotemporal Transformer, trained on CMIP6 multi-model ensembles and validated against ERA5 reanalysis, enabling global-scale, monthly-initialized forecasts of temperature and precipitation anomalies over lead times of up to four seasons. Key contributions include: (1) the first coupling of a variational autoencoder with a Transformer to jointly model latent climate dynamics and long-range spatiotemporal dependencies; (2) substantially improved forecast skill in teleconnection-dominated regions (e.g., mid- to high-latitude continental Europe) after detrending, outperforming the operational system SEAS5 for temperature and precipitation over select land areas; and (3) achieving operational-level latency alongside intrinsic interpretability.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Most operational climate services providers base their seasonal predictions on initialised general circulation models (GCMs) or statistical techniques that fit past observations. GCMs require substantial computational resources, which limits their capacity. In contrast, statistical methods often lack robustness due to short historical records. Recent works propose machine learning methods trained on climate model output, leveraging larger sample sizes and simulated scenarios. Yet, many of these studies focus on prediction tasks that might be restricted in spatial extent or temporal coverage, opening a gap with existing operational predictions. Thus, the present study evaluates the effectiveness of a methodology that combines variational inference with transformer models to predict fields of seasonal anomalies. The predictions cover all four seasons and are initialised one month before the start of each season. The model was trained on climate model output from CMIP6 and tested using ERA5 reanalysis data. We analyse the method's performance in predicting interannual anomalies beyond the climate change-induced trend. We also test the proposed methodology in a regional context with a use case focused on Europe. While climate change trends dominate the skill of temperature predictions, the method presents additional skill over the climatological forecast in regions influenced by known teleconnections. We reach similar conclusions based on the validation of precipitation predictions. Despite underperforming SEAS5 in most tropics, our model offers added value in numerous extratropical inland regions. This work demonstrates the effectiveness of training generative models on climate model output for seasonal predictions, providing skilful predictions beyond the induced climate change trend at time scales and lead times relevant for user applications.

Problem

• Combines variational inference and transformers for seasonal climate predictions
• Evaluates performance in predicting interannual anomalies beyond climate trends
• Tests methodology in regional context with focus on Europe

Innovation

• Combines variational inference with transformers
• Predicts seasonal anomalies globally
• Trained on CMIP6 climate data