LSKNet: A Foundation Lightweight Backbone for Remote Sensing

2024-03-18

International Journal of Computer Vision

AI Summary

Remote sensing image recognition faces dual challenges: insufficient long-range contextual modeling and redundant fixed receptive fields, while existing methods often neglect scene-level prior knowledge, leading to misclassification. To address these issues, we propose LSKNet—a lightweight backbone network incorporating Large Selective Kernel (LSK) mechanisms. This work is the first to introduce the LSK mechanism into remote sensing, explicitly encoding object-scale priors via dynamically adjustable large spatial receptive fields. LSKNet further integrates channel-spatial adaptive weighting with multi-scale contextual modeling. Crucially, it achieves significant improvements in long-range dependency modeling without increasing computational overhead. Extensive experiments demonstrate state-of-the-art performance on standard benchmarks across three fundamental remote sensing tasks: image classification, object detection, and semantic segmentation—achieving an optimal balance between accuracy and efficiency.

Technology Category

• Computer Vision: Remote Sensing / Geospatial AI
• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Remote sensing images pose distinct challenges for downstream tasks due to their inherent complexity. While a considerable amount of research has been dedicated to remote sensing classification, object detection and semantic segmentation, most of these studies have overlooked the valuable prior knowledge embedded within remote sensing scenarios. Such prior knowledge can be useful because remote sensing objects may be mistakenly recognized without referencing a sufficiently long-range context, which can vary for different objects. This paper considers these priors and proposes a lightweight Large Selective Kernel Network (LSKNet) backbone. LSKNet can dynamically adjust its large spatial receptive field to better model the ranging context of various objects in remote sensing scenarios. To our knowledge, large and selective kernel mechanisms have not been previously explored in remote sensing images. Without bells and whistles, our lightweight LSKNet sets new state-of-the-art scores on standard remote sensing classification, object detection and semantic segmentation benchmarks. Our comprehensive analysis further validated the significance of the identified priors and the effectiveness of LSKNet. The code is available at https://github.com/zcablii/LSKNet.

Problem

• Addresses complexity in remote sensing image tasks
• Incorporates prior knowledge for object recognition
• Proposes lightweight LSKNet for dynamic context modeling

Innovation

• Lightweight Large Selective Kernel Network
• Dynamic spatial receptive field adjustment
• State-of-the-art remote sensing benchmarks