DINOv3 as a Frozen Encoder for CRPS-Oriented Probabilistic Rainfall Nowcasting

2025-11-14

AI Summary

This work addresses probabilistic nowcasting of 4-hour accumulated precipitation. We propose an efficient end-to-end framework that freezes the pre-trained satellite vision encoder DINOv3-SAT493M to extract spatiotemporal features, followed by a lightweight video projector (V-JEPA ViT) and a discrete empirical cumulative distribution function (eCDF) probabilistic head, optimized solely via the Continuous Ranked Probability Score (CRPS). Our key innovations include: (i) freezing large-model parameters to decouple representation learning from probabilistic modeling; and (ii) replacing conventional parametric distribution assumptions (e.g., Gamma-Hurdle) with explicit eCDF-based modeling. Evaluated on the Weather4Cast 2025 benchmark, our method achieves a CRPS of 3.5102—26% lower than the best-performing 3D-UNet—demonstrating substantial improvements in both forecast accuracy and computational efficiency.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Probabilistic Circuits and Graphical Models
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

This paper proposes a competitive and computationally efficient approach to probabilistic rainfall nowcasting. A video projector (V-JEPA Vision Transformer) associated to a lightweight probabilistic head is attached to a pre-trained satellite vision encoder (DINOv3 ext{-}SAT493M) to map encoder tokens into a discrete empirical CDF (eCDF) over 4-hour accumulated rainfall. The projector-head is optimized end-to-end over the Continuous Ranked Probability Score (CRPS). As an alternative, 3D-UNET baselines trained with an aggregate Rank Probability Score and a per-pixel Gamma-Hurdle objective are used. On the Weather4Cast 2025 benchmark, the proposed method achieved a promising performance, with a CRPS of 3.5102 (CRPS), which represents $approx$26% in effectiveness gain against the best 3D-UNET.

Problem

• Develops probabilistic rainfall nowcasting using frozen DINOv3 encoder
• Maps satellite tokens to rainfall CDF via video projector
• Optimizes CRPS score for 4-hour accumulated rainfall prediction

Innovation

• DINOv3 as frozen satellite vision encoder
• V-JEPA projector with lightweight probabilistic head
• End-to-end optimization using CRPS objective