A Controlled Visual-Backbone Benchmark for Multimodal Short-Term Solar Irradiance Forecasting

2026-07-26

AI Summary

This study addresses the challenge of fairly evaluating the contribution of visual backbone networks in solar irradiance forecasting, as existing approaches often modify multiple components simultaneously. To enable reproducible and equitable comparisons, the authors propose a controlled benchmarking protocol that isolates the visual encoder by fixing all other elements of the multimodal prediction pipeline—including historical weather encoding, clear-sky index normalization, and the fusion and regression heads—while only swapping the visual backbone. Experiments on the Folsom and NREL datasets validate the effectiveness of this approach: all tested backbones (ConvNeXt, Swin Transformer, VMamba, Spatial Mamba, and MambaVision) outperform the smart persistence baseline. Notably, VMamba Small and Swin Base achieve the lowest RMSE (~65.4 W/m²) on Folsom, while Swin Tiny yields the best performance on NREL (23.76 W/m²).

Abstract

Sky-image irradiance studies often compare forecasting systems in which the image encoder, temporal model, fusion block, target definition, and training recipe all change together. We use a narrower protocol: the multimodal forecasting pipeline is fixed, and only the visual backbone is varied. The shared setup keeps preprocessing, clear-sky-index normalization, weather-history encoding, fusion, regression head, loss, optimizer schedule, seed, and chronological split policy unchanged. We compare ConvNeXt, Swin Transformer, VMamba, Spatial Mamba, and MambaVision backbones for 10min-ahead forecasting on Folsom and a strict matched NREL split. Forecast skill is measured against clear-sky-index smart persistence, and temporal-only rows are reported as weather-history diagnostics rather than as the main ranking criterion. On the Folsom strict split, all evaluated visual-backbone runs improve over smart persistence. In the evaluated single-seed strict runs, VMamba Small and Swin Base reach matched Folsom RMSE values of 65.39 W/m^2 and 65.50 W/m^2; the temporal-only diagnostic reaches 69.51 W/m^2. On the 313-sample NREL strict split, smart persistence remains strongest at 17.48 W/m^2, while the lowest visual RMSE is obtained by Swin Tiny at 23.76 W/m^2. These results provide a reproducible encoder comparison under one fixed multimodal operating point rather than establishing architecture-level dominance, statistically resolved ranking, or fully optimized forecasting performance. Code available here: https://github.com/Oshadha345/irradiance_benchmark

Problem

• solar irradiance forecasting
• visual backbone
• multimodal benchmark
• short-term forecasting
• sky-image

Innovation

• visual backbone benchmark
• multimodal solar forecasting
• controlled evaluation protocol
• short-term irradiance prediction
• encoder comparison