multisensor reflectance harmonization

To address the limited color correction accuracy of mobile device cameras constrained by single-modality RGB input, this paper proposes an end-to-end jointly optimized framework that fuses high-resolution RGB with low-resolution multispectral sensor data. Unlike conventional approaches relying on hand-crafted priors or feature concatenation, our method preserves the full multispectral information flow throughout the pipeline and unifies the modeling of sensor response, spectral reconstruction, and color mapping—enabling seamless integration with state-of-the-art image architectures. Trained on a custom multispectral rendering dataset, the model achieves computational efficiency while significantly improving cross-device robustness. Experiments demonstrate a reduction of up to 50% in mean color error (ΔE₀₀) compared to the best RGB-only baseline, outperforming methods using multispectral priors alone, and exhibiting superior stability across diverse hardware spectral responses.

Existing spectral response modeling for digital cameras is typically confined to isolated components, lacking an end-to-end, physically consistent description from illumination input to pixel intensity output—thus limiting color fidelity and spectral accuracy. This paper introduces the first full-chain, physics-driven end-to-end spectral–color joint modeling framework. It unifies the coupled effects of optical system transmission, sensor quantum efficiency, color filter array (CFA) spectral transmittance, and nonlinear pixel response. The model integrates empirically measured RGB camera spectral responses with data-driven nonlinear mapping correction. Evaluated under multiple illuminants, it achieves superior color reproduction (mean ΔE < 1.2) and significantly improved spectral reconstruction fidelity (37% reduction in RMSE). Validated across machine vision, remote sensing, and computational spectral imaging applications, this work bridges a critical theoretical and practical gap in end-to-end camera spectral response modeling.

Existing remote sensing foundation models are constrained by fixed spectral band configurations and spatial resolutions, limiting generalization to practical scenarios involving band missingness, cross-sensor fusion, and unseen spatial scales. To address this, we propose RSFM—the first optical remote sensing foundation model supporting arbitrary band combinations, sensor types, and spatial resolutions. Our approach introduces four key innovations: (1) a spectrum-agnostic tokenizer; (2) multi-scale adaptive image patch embedding; (3) a multi-scale semantic alignment mechanism; and (4) a channel-level masked autoencoding pretraining strategy—enabling joint spectral-spatial modeling and dynamic resolution adaptation. Evaluated across 10+ benchmark datasets—including Sentinel-2, Landsat, and HLS—RSFM achieves state-of-the-art performance on band-missing, cross-sensor, and cross-resolution transfer tasks, significantly enhancing universal representation learning for heterogeneous, multi-source remote sensing data.

Existing hyperspectral image fusion methods suffer from low spectral fidelity, spatial detail distortion, insufficient noise suppression, and inconsistent evaluation protocols—hindering algorithm development and fair benchmarking. To address these issues, this work introduces HSFusion-Bench: the first open-source, reproducible PyTorch-based benchmarking toolbox for hyperspectral pansharpening. It integrates 12 state-of-the-art methods, a standardized multi-source dataset, and a comprehensive full-reference evaluation framework (including QNR, ERGAS, SAM, etc.). For the first time, it systematically uncovers the fundamental trade-off among spectral fidelity, spatial resolution, and computational efficiency. Quantitative analysis reveals common deficiencies of existing methods under complex imaging conditions. HSFusion-Bench has become the de facto standard in the field, significantly improving the efficiency of algorithm development, validation, and comparative analysis, and enabling multiple follow-up studies.

To address the challenges of inconsistent feature representations across remote sensing sensors (e.g., Sentinel-2 and aerial imagery) and limited high-resolution annotated data—leading to poor cross-resolution generalization—this paper proposes X-STARS, a cross-sensor self-supervised training and alignment framework. Its core innovation is the first multi-sensor alignment dense loss, which achieves cross-resolution and cross-platform feature alignment via contrastive image-patch matching. X-STARS supports both from-scratch pretraining and continual pretraining paradigms. Evaluated on our newly constructed Cities-France multi-sensor dataset, X-STARS consistently outperforms state-of-the-art methods across seven downstream classification and segmentation tasks. Moreover, it achieves comparable performance using 30–50% fewer annotated samples, significantly reducing annotation burden while enhancing model transferability across heterogeneous remote sensing modalities.

This study addresses a critical gap in remote sensing research—the lack of large-scale, physically consistent synthetic datasets that combine high spectral resolution with pixel-level ground truth for vegetation traits. To this end, we integrate the PROSAIL radiative transfer model with Sentinel-2 Level-2A inversion results to generate, for the first time, physically realistic hyperspectral image cubes (400–2500 nm, 64×64 pixels) across four ecologically distinct regions, yielding 10,915 samples. Each sample includes paired pixel-level vegetation trait maps, uncertainty bounds, and scene classification layers. The resulting dataset enables rapid radiative transfer simulations, benchmarking of inversion algorithms, and investigations into spectral–biophysical relationships, offering a high-quality, verifiable reference resource for advancing remote sensing modeling and machine learning applications.

This study addresses geometrically induced radiometric inconsistencies in low-altitude UAV multispectral imagery, which arise from wide fields of view and multi-angle observations. The authors propose a geometry-aware method to systematically extract multi-angle reflectance measurements and corresponding viewing geometry for the same ground target from UAV multispectral data for the first time. By refining camera interior and exterior parameters via structure-from-motion (SfM), homogeneous regions identified in orthomosaics are back-projected onto original multi-view sub-images, enabling joint retrieval of ten-band reflectance values and observation angles, and facilitating anisotropic reflectance visualization. Experimental results demonstrate that reflectance extremes for grass targets differ by 119%–137% in the red-edge and near-infrared bands, providing strong evidence of the significant impact of observation geometry on radiometric consistency.

This work addresses the lack of paired long-wave infrared (LWIR) hyperspectral imagery and ground-truth temperature–emissivity–texture (TeX) annotations that has hindered learning-based TeX decomposition. To this end, we introduce TeX-1500, the first large-scale, real-world paired dataset comprising 1,522 radiometrically calibrated and wavelength-aligned LWIR hyperspectral images along with their consistently reconstructed TeX ground truth, spanning diverse locations, seasons, times of day, and sensor types. Leveraging this dataset, we propose a wavelength-aware TeX-UNet baseline model and demonstrate its effectiveness on the DARPA IH benchmark as well as zero- and few-shot transfer tasks using FTIR data. Our contribution establishes the first reproducible supervised benchmark for physics-driven thermal perception research.

Existing foundation models for Earth observation struggle to effectively incorporate hyperspectral imagery (HSI), while specialized HSI models lack joint pretraining with multimodal remote sensing data. This work proposes a hierarchical Transformer architecture that, for the first time, enables unified pretraining of HSI alongside multispectral and SAR data through spectral tokenization, sensor-specific encoders, and a cross-sensor fusion module. The authors also introduce SpectralEarth-MM, a large-scale co-located multimodal dataset. Leveraging a JEPA-style joint embedding prediction objective, the model achieves state-of-the-art performance on both hyperspectral downstream tasks and general Earth observation benchmarks, significantly enhancing its generalization and multimodal fusion capabilities.

本文提出一种传感器级流水线，通过校正、配准、裁剪和编码方法解决RGB与热图像几何差异问题，以提高无人机立面异常检测精度。