PIM: Physics-Informed Multi-task Pre-training for Improving Inertial Sensor-Based Human Activity Recognition

2025-03-23

AI Summary

To address the scarcity of labeled data in inertial measurement unit (IMU)-based human activity recognition (HAR), this paper proposes a physics-mechanism-driven multi-task self-supervised pretraining framework. It systematically formulates fundamental biomechanical principles—including dynamic constraints, kinematic continuity, and left-right limb geometric symmetry—as learnable pretext tasks, integrating physics-informed feature extraction, contrastive learning, and signal reconstruction in a joint optimization objective. The method learns physically consistent representations without requiring manual annotations. Evaluated on four benchmark IMU-HAR datasets, it achieves near 10% improvement in macro-F1 score and accuracy under few-shot settings (2–8 labeled samples per class), and still yields a 3% gain under full supervision—significantly outperforming existing state-of-the-art methods. This work effectively bridges the gap between physical priors and multi-sensor representation learning.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Computer Vision: Low Level & Physics-based Vision
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Human activity recognition (HAR) with deep learning models relies on large amounts of labeled data, often challenging to obtain due to associated cost, time, and labor. Self-supervised learning (SSL) has emerged as an effective approach to leverage unlabeled data through pretext tasks, such as masked reconstruction and multitask learning with signal processing-based data augmentations, to pre-train encoder models. However, such methods are often derived from computer vision approaches that disregard physical mechanisms and constraints that govern wearable sensor data and the phenomena they reflect. In this paper, we propose a physics-informed multi-task pre-training (PIM) framework for IMU-based HAR. PIM generates pre-text tasks based on the understanding of basic physical aspects of human motion: including movement speed, angles of movement, and symmetry between sensor placements. Given a sensor signal, we calculate corresponding features using physics-based equations and use them as pretext tasks for SSL. This enables the model to capture fundamental physical characteristics of human activities, which is especially relevant for multi-sensor systems. Experimental evaluations on four HAR benchmark datasets demonstrate that the proposed method outperforms existing state-of-the-art methods, including data augmentation and masked reconstruction, in terms of accuracy and F1 score. We have observed gains of almost 10% in macro f1 score and accuracy with only 2 to 8 labeled examples per class and up to 3% when there is no reduction in the amount of training data.

Problem

• Improves HAR using physics-informed multi-task pre-training
• Addresses lack of labeled data via SSL with physical constraints
• Enhances accuracy by modeling human motion physics in IMU data

Innovation

• Physics-informed multi-task pre-training framework
• Leverages physical aspects of human motion
• Outperforms state-of-the-art methods significantly