🤖 AI Summary
To address the challenge of deploying deep models in edge remote sensing scenarios—such as satellite and UAV platforms—where low power consumption, real-time inference, and online environmental adaptation (e.g., weather variations, sensor drift) are simultaneously required, this paper proposes the first spiking neural network (SNN)-based online adaptive framework tailored for remote sensing tasks. Methodologically, we introduce a forward temporal approximate backpropagation through time (BPTT) algorithm, an adaptive activation scaling (AAS) mechanism, and a confidence-weighted instance learning strategy, collectively enhancing classification, segmentation, and detection performance under low-timestep constraints. Evaluated on seven remote sensing benchmark datasets, our framework consistently outperforms state-of-the-art domain adaptation and generalization methods, achieving millisecond-level dynamic responsiveness, over 60% measured power reduction, and efficient on-device deployment.
📝 Abstract
On-device computing, or edge computing, is becoming increasingly important for remote sensing, particularly in applications like deep network-based perception on on-orbit satellites and unmanned aerial vehicles (UAVs). In these scenarios, two brain-like capabilities are crucial for remote sensing models: 1) high energy efficiency, allowing the model to operate on edge devices with limited computing resources, and 2) online adaptation, enabling the model to quickly adapt to environmental variations, weather changes, and sensor drift. This work addresses these needs by proposing an online adaptation framework based on spiking neural networks (SNNs) for remote sensing. Starting with a pretrained SNN model, we design an efficient, unsupervised online adaptation algorithm, which adopts an approximation of the backpropagation through time (BPTT) algorithm and only involves forward-in-time computation that significantly reduces the computational complexity of SNN adaptation learning. Besides, we propose an adaptive activation scaling (AAS) scheme to boost online SNN adaptation performance, particularly in low time-steps. Furthermore, for the more challenging remote sensing detection task, we propose a confidence-based instance weighting scheme, which substantially improves adaptation performance in the detection task. To our knowledge, this work is the first to address the online adaptation of SNNs. Extensive experiments on seven benchmark datasets across classification, segmentation, and detection tasks demonstrate that our proposed method significantly outperforms existing domain adaptation and domain generalization approaches under varying weather conditions. The proposed method enables energy-efficient and fast online adaptation on edge devices, and has much potential in applications such as remote perception on on-orbit satellites and UAVs. The code is available at https://github.com/ThunderDavid/OASNN.