SDHSI-Net: Learning Better Representations for Hyperspectral Images via Self-Distillation

2026-01-12

AI Summary

This work addresses the challenges of hyperspectral image classification, where high-dimensional features and limited labeled data often lead to overfitting and excessive computational costs. To mitigate these issues, the study introduces a self-distillation mechanism—applied for the first time in this context—that leverages early model outputs as soft targets to enhance consistency between intermediate and final predictions. This strategy improves intra-class compactness and inter-class separability in the feature space. By integrating deep convolutional networks with joint spectral-spatial learning, the proposed method achieves significant gains in classification accuracy and model robustness without requiring an external teacher network. Extensive experiments on two benchmark datasets demonstrate its superior performance compared to existing approaches.

Technology Category

• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Computer Vision: Representation Learning for Vision
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Hyperspectral image (HSI) classification presents unique challenges due to its high spectral dimensionality and limited labeled data. Traditional deep learning models often suffer from overfitting and high computational costs. Self-distillation (SD), a variant of knowledge distillation where a network learns from its own predictions, has recently emerged as a promising strategy to enhance model performance without requiring external teacher networks. In this work, we explore the application of SD to HSI by treating earlier outputs as soft targets, thereby enforcing consistency between intermediate and final predictions. This process improves intra-class compactness and inter-class separability in the learned feature space. Our approach is validated on two benchmark HSI datasets and demonstrates significant improvements in classification accuracy and robustness, highlighting the effectiveness of SD for spectral-spatial learning. Codes are available at https://github.com/Prachet-Dev-Singh/SDHSI.

Problem

• Hyperspectral image classification
• high spectral dimensionality
• limited labeled data
• overfitting
• computational cost

Innovation

• Self-Distillation
• Hyperspectral Image Classification
• Spectral-Spatial Learning
• Soft Targets
• Feature Compactness