WiFuse: An Attention Mechanism for Human Activity Recognition using Fused CSI Amplitude and Delay-Doppler Channel Features

2026-08-01

AI Summary

Wi-Fi signals are highly susceptible to multipath effects and noise interference, which degrade the performance of channel state information (CSI)-based human activity recognition. To address this, this work proposes WiFuse, a novel framework that, for the first time, integrates denoised CSI amplitude variations and phase-derived Delay-Doppler two-dimensional motion representations within a dual-stream architecture. The framework combines ResNet with a temporal convolutional network (TCN) and incorporates both channel-wise and spatiotemporal attention mechanisms to enhance feature learning. Furthermore, a two-stage decoupled transfer learning strategy is introduced to mitigate domain shift and class overlap challenges. Evaluated on the XRF55 and Wi-MIR datasets, WiFuse achieves recognition accuracies of 95.28% and 98.20%, respectively, significantly outperforming existing state-of-the-art methods.

Abstract

Recently, Wi-Fi sensing has played a significant role in Human Activity Recognition (HAR), as it enables the detection of various activities using only Wi-Fi signals, ensuring privacy and remaining non-intrusive for the user. However, environmental characteristics such as reflective surfaces, hardware offsets, and other physical impairments affect recognition by the neural network, subsequently causing errors and significantly reducing model accuracy. To overcome this problem we present the WiFuse framework, a dual-stream Channel State Information (CSI) framework for human activity recognition (HAR) that pairs denoised time-domain amplitude variations with 2D-FFT-derived Delay-Doppler motion representations computed from the sanitized channel phase. The fused representation feeds a hybrid ResNet-Temporal Convolutional Network (TCN) neural architecture augmented with channel and spatio-temporal attention, where the ResNet extracts spatial-spectral features and the TCN models long-range temporal dependencies; a decoupled two-stage transfer learning strategy is employed to improve optimization stability and feature reuse. We conduct extensive experiments on two public datasets, including comparisons against state-of-the-art methods and alternative hybrid architectures, ablation studies, and cross-dataset and domain-adaptation evaluations. The proposed framework reaches an overall accuracy of up to 95.28% across the four environments of the XRF55 dataset and up to 98.20% on the multi-user Wi-MIR dataset. Overall, the results indicate that combining amplitude and Delay-Doppler representations within a dual-stream strategy, enhanced by transfer learning, improves recognition performance under conditions that typically degrade deep neural networks, such as class overlap, multipath propagation, noise, and interference.

Problem

• Human Activity Recognition
• Wi-Fi sensing
• Channel State Information
• environmental interference
• multipath propagation

Innovation

• Dual-stream fusion
• Delay-Doppler representation
• Channel State Information (CSI)
• Attention mechanism
• Transfer learning