EnergyFormer: Energy Attention with Fourier Embedding for Hyperspectral Image Classification

2025-03-11

AI Summary

To address challenges in hyperspectral image (HSI) analysis—including high-dimensional spectral redundancy, significant spectral variability, and difficulty modeling long-range spectral–spatial dependencies—this paper proposes an end-to-end Transformer-based framework. Its key contributions are: (1) a Multi-Head Energy Attention (MHEA) mechanism that enhances discriminability of spectral responses; (2) Fourier Position Encoding (FoPE), enabling adaptive modeling of long-range spectral–spatial dependencies; and (3) an Enhanced Convolutional Block Attention Module (ECBAM) to improve band-wise selectivity and structural awareness. Evaluated on WHU-Hi-HanChuan, Salinas, and PaviaU datasets, the method achieves overall accuracies of 99.28%, 98.63%, and 98.72%, respectively—substantially outperforming CNNs, standard Transformers, and Mamba-based baselines. These results demonstrate the framework’s effectiveness and state-of-the-art performance for fine-grained HSI classification.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Hyperspectral imaging (HSI) provides rich spectral-spatial information across hundreds of contiguous bands, enabling precise material discrimination in applications such as environmental monitoring, agriculture, and urban analysis. However, the high dimensionality and spectral variability of HSI data pose significant challenges for feature extraction and classification. This paper presents EnergyFormer, a transformer-based framework designed to address these challenges through three key innovations: (1) Multi-Head Energy Attention (MHEA), which optimizes an energy function to selectively enhance critical spectral-spatial features, improving feature discrimination; (2) Fourier Position Embedding (FoPE), which adaptively encodes spectral and spatial dependencies to reinforce long-range interactions; and (3) Enhanced Convolutional Block Attention Module (ECBAM), which selectively amplifies informative wavelength bands and spatial structures, enhancing representation learning. Extensive experiments on the WHU-Hi-HanChuan, Salinas, and Pavia University datasets demonstrate that EnergyFormer achieves exceptional overall accuracies of 99.28%, 98.63%, and 98.72%, respectively, outperforming state-of-the-art CNN, transformer, and Mamba-based models. The source code will be made available at https://github.com/mahmad000.

Problem

• High dimensionality and spectral variability in HSI data challenge feature extraction.
• EnergyFormer enhances spectral-spatial feature discrimination using transformer-based innovations.
• Proposed method improves classification accuracy on multiple HSI datasets.

Innovation

• Multi-Head Energy Attention optimizes feature discrimination
• Fourier Position Embedding encodes spectral-spatial dependencies
• Enhanced Convolutional Block Attention Module amplifies informative features