Detecting Informative Channels: ActionFormer

2025-05-27

AI Summary

ActionFormer struggles to model high-order temporal dynamics and spatiotemporal feature coupling in inertial measurement unit (IMU) signal-based human activity recognition. To address this, we propose an improved architecture specifically designed for IMU time-series data. Our method adapts ActionFormer to IMU signals for the first time, introduces a Sequence-and-Excitation module for channel-wise adaptive recalibration while constraining parameter growth, and adopts the Swish activation function to preserve negative-gradient information—enhancing discriminability at subtle action boundaries. The approach performs end-to-end modeling of raw sensor sequences by integrating sequence-level attention with lightweight channel modulation. Evaluated on the WEAR dataset, our method achieves a 16.01% absolute improvement in mean average precision (mAP), demonstrating substantial gains in both boundary localization and classification accuracy for short-duration, fine-grained actions—e.g., “tapping” and “fist-clenching”.

Technology Category

• Planning, Routing, and Scheduling: Activity and Plan Recognition
• Computer Vision: Video Understanding & Activity Analysis
• Knowledge Representation and Reasoning: Action, Change, and Causality

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Human Activity Recognition (HAR) has recently witnessed advancements with Transformer-based models. Especially, ActionFormer shows us a new perspectives for HAR in the sense that this approach gives us additional outputs which detect the border of the activities as well as the activity labels. ActionFormer was originally proposed with its input as image/video. However, this was converted to with its input as sensor signals as well. We analyze this extensively in terms of deep learning architectures. Based on the report of high temporal dynamics which limits the model's ability to capture subtle changes effectively and of the interdependencies between the spatial and temporal features. We propose the modified ActionFormer which will decrease these defects for sensor signals. The key to our approach lies in accordance with the Sequence-and-Excitation strategy to minimize the increase in additional parameters and opt for the swish activation function to retain the information about direction in the negative range. Experiments on the WEAR dataset show that our method achieves substantial improvement of a 16.01% in terms of average mAP for inertial data.

Problem

• Detecting informative channels in human activity recognition
• Improving ActionFormer for sensor signal inputs
• Addressing temporal dynamics and feature interdependencies in HAR

Innovation

• Modified ActionFormer for sensor signals
• Sequence-and-Excitation strategy minimizes parameters
• Swish activation retains negative range information