HyperSAM: A Promptable Foundation Model for Hyperspectral Remote Sensing

2026-09-29

AI Summary

This study addresses the scarcity of annotated data and the underutilization of visual priors in hyperspectral foundation models by proposing a promptable hyperspectral foundation model. Methodologically, a physics-informed full-spectrum synthesis pipeline is constructed to generate high-quality training data. Architecturally built upon SAM3 and ViT, the model introduces a mixture-of-experts fine-tuning mechanism that integrates a frozen RGB branch with zero-initialized injection. Efficient spectral adaptation is further achieved through ControlNet-style feature injection and cross-modal sample selection. Experimental results demonstrate that the proposed model exhibits strong generalization across diverse tasks, including classification and anomaly detection, validating that high-quality synthetic data outperforms noisy supervision scaling strategies.

Abstract

Hyperspectral remote sensing provides dense spectral measurements that are indispensable for material-level Earth observation, yet the construction of a general-purpose hyperspectral foundation model remains difficult. Two bottlenecks are especially limiting. First, large hyperspectral corpora rarely provide high spatial resolution together with reliable dense annotations. Second, many hyperspectral models are still trained almost from scratch, so the geometric and interactive priors learned by modern vision foundation models are not fully reused. To alleviate these issues, we \highlight{present} \textbf{HyperSAM}, a promptable hyperspectral foundation model that couples a data-centric hyperspectral synthesis pipeline with a spectral adaptation architecture based on Segment Anything Model 3 (SAM3). On the data side, HyperSAM synthesizes full-spectrum hyperspectral cubes from high-resolution SpaceNet multispectral imagery through a physics-informed abundance-transfer generator, while SAM3-derived pseudo-masks provide object-centric supervision. On the model side, the latest implementation uses a frozen SAM3 RGB image branch, a trainable hyperspectral side encoder initialized from the RGB vision transformer (ViT), ControlNet-style zero-initialized feature injection, and a lightweight mixture-of-experts mask refiner. To enhance training robustness against noisy pseudo-labels, Cross-modal Sample Selection (CromSS)-style confidence selection is incorporated for noisy-label weighting. Extensive experiments show that HyperSAM obtains strong generalization on diverse hyperspectral tasks (e.g., classification, anomaly detection, change detection, target detection, and airborne oil-spill mapping) and that high-quality synthetic hyperspectral data can be more effective than simply scaling noisy hyperspectral supervision.

Problem

• Hyperspectral remote sensing
• Foundation model
• Dense annotations
• Spatial resolution
• Prior knowledge

Innovation

• Hyperspectral Foundation Model
• Segment Anything Model 3 (SAM3)
• Physics-informed Synthesis
• Zero-initialized Feature Injection
• Noisy-label Robustness