Multi Image Super Resolution Modeling for Earth System Models

2025-02-18

AI Summary

To address the insufficient spatial detail in low-resolution remote sensing and climate simulation data from Earth system models (ESMs), this paper proposes a physically consistent end-to-end super-resolution (SR) method. The approach innovatively embeds Fourier basis activation functions into a Vision Transformer (ViT), synergistically combining ViT’s global contextual modeling capability with the high-frequency representational power of implicit neural representations (INRs). A multi-scale loss function is introduced to enhance physical interpretability and structural fidelity of reconstructions. Evaluated on temperature and radiative flux data, the method achieves up to a 4.18 dB improvement in PSNR over state-of-the-art baselines—including standard ViT, SIREN, and SRGAN—demonstrating superior balance between global structural coherence and local texture detail while preserving physical consistency.

Technology Category

• Computer Vision: Low Level & Physics-based Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures

Abstract

Super-resolution (SR) techniques are essential for improving Earth System Model (ESM) data's spatial resolution, which helps better understand complex environmental processes. This paper presents a new algorithm, ViFOR, which combines Vision Transformers (ViT) and Implicit Neural Representation Networks (INRs) to generate High-Resolution (HR) images from Low-Resolution (LR) inputs. ViFOR introduces a novel integration of Fourier-based activation functions within the Vision Transformer architecture, enabling it to effectively capture global context and high-frequency details critical for accurate SR reconstruction. The results show that ViFOR outperforms state-of-the-art methods such as ViT, Sinusoidal Representation Networks (SIREN), and SR Generative Adversarial Networks (SRGANs) based on metrics like Peak Signal-to-Noise Ratio (PSNR) and Mean Squared Error (MSE) both for global as well as the local imagery. ViFOR improves PSNR of up to 4.18 dB, 1.56 dB, and 1.73 dB over ViT for full images in the Source Temperature, Shortwave, and Longwave Flux.

Problem

• Enhance Earth System Model resolution
• Develop ViFOR for super-resolution imaging
• Improve global and local image accuracy

Innovation

• Combines Vision Transformers and INRs
• Integrates Fourier-based activation functions
• Outperforms ViT, SIREN, and SRGANs