Regularization of ML models for Earth systems by using longer model timesteps

2025-03-23

AI Summary

Machine learning models for chaotic Earth system modeling suffer from poor generalization and overconfident predictions. Method: This paper proposes elongating the model’s temporal step size as an implicit regularization mechanism. Theoretical analysis first reveals that long time steps inherently induce input–output perturbations in chaotic systems, effectively serving as a hyperparameter-free robust regularizer. We further develop an adaptive step-size selection strategy grounded in multiscale temporal analysis. Results: Evaluated on the ORAS5 ocean reanalysis dataset, a 28-day step size significantly enhances simulation fidelity and cross-temporal-spatial generalization. The approach is simple, broadly applicable, and yields an interpretable, hyperparameter-free regularization paradigm for Earth system ML modeling—bridging dynamical understanding with statistical learning.

Technology Category

• Machine Learning: Scalability of ML Systems
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: Studies of user behavior, including longitudinal effects of personalized systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Regularization is a technique to improve generalization of machine learning (ML) models. A common form of regularization in the ML literature is to train on data where similar inputs map to different outputs. This improves generalization by preventing ML models from becoming overconfident in their predictions. This paper shows how using longer timesteps when modelling chaotic Earth systems naturally leads to more of this regularization. We show this in two domains. We explain how using longer model timesteps can improve results and demonstrate that increased regularization is one of the causes. We explain why longer model timesteps lead to improved regularization in these systems and present a procedure to pick the model timestep. We also carry out a benchmarking exercise on ORAS5 ocean reanalysis data to show that a longer model timestep (28 days) than is typically used gives realistic simulations. We suggest that there will be many opportunities to use this type of regularization in Earth system problems because the Earth system is chaotic and the regularization is so easy to implement.

Problem

• Improving ML generalization in Earth systems via longer timesteps
• Exploring regularization benefits from chaotic system timestep adjustments
• Optimizing model timestep selection for realistic Earth system simulations

Innovation

• Longer timesteps regularize chaotic Earth systems
• Procedure to optimize model timestep selection
• Benchmarked 28-day timestep improves ocean simulations