Timestep-Conditioned Transformers for Global Weather Forecasting

2026-08-06

AI Summary

This work proposes GEM-3, the first probabilistic global weather forecasting model capable of supporting arbitrary time-step configurations during inference, addressing the longstanding trade-off between high-resolution short-term predictions and error accumulation in long-range forecasts inherent to fixed time-step models. By employing joint multi-time-step training, GEM-3 adaptively balances performance across diverse forecast horizons. Built upon a lightweight neighborhood-attention Transformer architecture with 134 million parameters and deployed on an equirectangular grid, the model significantly enhances the stability of long-duration rollouts. Compared to its predecessor GEM-2, GEM-3 maintains near state-of-the-art skill in medium-range probabilistic forecasting while offering improved computational efficiency and enhanced diagnostic capabilities for decision support.

Abstract

Existing machine-learning weather forecasting models rely on predetermined and fixed autoregressive timesteps. The choice of model timestep involves a fundamental trade-off: shorter timesteps (e.g. 1 to 6 hours) finely resolve atmospheric dynamics within the diurnal cycle but increase error accumulation for a given forecast horizon, while longer timesteps (e.g. 24 hours) reduce error accumulation but limit the usability of short-range forecasts where sub-daily predictability is high. In this work, we present GEM-3, a probabilistic global weather model that addresses this trade-off through explicit multi-timestep inference. With a single set of trained weights, the model timestep can be configured at inference time to balance predictability and usability across a broad forecast horizon. Additionally, we find that mixed-timestep training consistently improves rollout stability relative to timestep-specialist models. Under the hood, GEM-3 is a lightweight neighborhood-attention transformer with ~134M parameters on an equirectangular grid with a number of architectural advancements beyond its predecessor GEM-2. The result is a practical forecasting system that couples near-SOTA medium-range probabilistic skill, stable extended-range rollouts, efficient training and inference, and decision-relevant diagnostics.

Problem

• weather forecasting
• autoregressive timesteps
• error accumulation
• forecast horizon
• sub-daily predictability

Innovation

• multi-timestep inference
• probabilistic weather forecasting
• neighborhood-attention transformer
• mixed-timestep training
• dynamic timestep configuration