Problem
Research questions and friction points this paper is trying to address.
Skeleton-based Action Recognition
Entity Bias
Late Fusion
World Coordinate System
Normalization
De-biasing Skeleton-Based Action Recognition with Convex Hull Adaptive Shift
🤖 AI Summary
This study addresses the issue of entity bias in skeleton-based action recognition, where the configuration of the world coordinate system origin violates the independent and identically distributed assumption and limits model performance. To this end, we propose CHASE, a novel normalization method that introduces a convex hull-constrained adaptive shifting mechanism to ensure the origin remains within the skeleton’s convex hull, thereby guaranteeing convergence. Furthermore, it incorporates a pairwise distribution distance auxiliary objective with a sub-entity strategy to uniformly handle both single-subject and multi-subject interaction tasks. As a plug-and-play module, CHASE is compatible with multi-modal data such as bone and velocity representations and can be seamlessly integrated into various backbone networks. Extensive experiments across seven datasets demonstrate that CHASE effectively eliminates entity bias, substantially improving the accuracy and robustness of action recognition.
📝 Abstract
Skeleton sequences can represent both individual actions and multi-entity interactions, encompassing human bodies, hands, objects, and robots. Existing approaches to recognize skeleton-based actions and interactions usually adopt a late fusion strategy, which expects individuals are independent and identically distributed to train a robust weight-shared entity encoder. However, observed entity bias in various skeletal data violates this assumption, leading to suboptimal optimization of backbone models that might produce wrong recognition results. This bias arises from the world coordinate system’s initial configuration, where the choice of origin often creates bias in representation. To this end, we propose a Convex Hull Adaptive Shift based normalization method to reduce Entity bias (CHASE), improving performance across a variety of skeleton-based action and interaction recognition tasks. To adaptively apply plausible shifts to the input skeletons, we formulate a plug-and-play parameterized network that ensures the relocated world origin lies within the skeleton convex hull, which avoids non-convergence by limiting the search space. To further minimize entity bias, we incorporate an auxiliary objective that leverages pair-wise distribution distances to guide network optimization. To support both single- and multi-entity actions, we propose a sub-entity strategy that offers a consistent formulation for both scenarios. Moreover, CHASE demonstrates compatibility with various intra-skeleton modalities, such as bones and velocities, highlighting its adaptability. Essentially, our method works as a normalization approach to reduce entity bias, enabling subsequent classifiers to achieve improved recognition performance across diverse settings. Extensive experiments on seven datasets, including NTU RGB+D, NTU RGB+D 120, H2O, Assembly101, Collective Activity, Volleyball, and HARPER, consistently verify our approach by seamlessly integrating with various backbones and significantly boosting their performance. Our code is publicly available at https://github.com/Necolizer/CHASE.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Skeleton-based Action Recognition
Entity Bias
Convex Hull Adaptive Shift
Plug-and-play Normalization
Sub-entity Strategy
🔎 Similar Papers
No similar papers found.
