A Structure-aware and Motion-adaptive Framework for 3D Human Pose Estimation with Mamba

2025-07-26

AI Summary

Existing Mamba-based 3D human pose lifting methods flatten joint sequences into 1D token sequences, failing to preserve the intrinsic skeletal topology and neglecting the heterogeneous motion patterns across joints. To address these limitations, we propose a structural-aware and motion-adaptive Mamba framework. First, we design a structural-aware state fusion module that explicitly encodes kinematic dependencies among joints via graph-structured priors. Second, we introduce a motion-adaptive modulation mechanism that jointly performs joint-specific dynamic modeling at both feature and state levels. By decoupling representation learning from strict sequential ordering, our approach mitigates over-homogenization of motion dynamics. Extensive experiments demonstrate state-of-the-art performance on Human3.6M and MPI-INF-3DHP benchmarks, achieving superior accuracy while reducing computational overhead by up to 32% compared to prior Mamba variants—thus striking an effective balance between precision and efficiency.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Structured Learning
• Computer Vision: Motion & Tracking

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Recent Mamba-based methods for the pose-lifting task tend to model joint dependencies by 2D-to-1D mapping with diverse scanning strategies. Though effective, they struggle to model intricate joint connections and uniformly process all joint motion trajectories while neglecting the intrinsic differences across motion characteristics. In this work, we propose a structure-aware and motion-adaptive framework to capture spatial joint topology along with diverse motion dynamics independently, named as SAMA. Specifically, SAMA consists of a Structure-aware State Integrator (SSI) and a Motion-adaptive State Modulator (MSM). The Structure-aware State Integrator is tasked with leveraging dynamic joint relationships to fuse information at both the joint feature and state levels in the state space, based on pose topology rather than sequential state transitions. The Motion-adaptive State Modulator is responsible for joint-specific motion characteristics recognition, thus applying tailored adjustments to diverse motion patterns across different joints. Through the above key modules, our algorithm enables structure-aware and motion-adaptive pose lifting. Extensive experiments across multiple benchmarks demonstrate that our algorithm achieves advanced results with fewer computational costs.

Problem

• Model intricate joint connections in 3D pose estimation
• Address diverse motion dynamics uniformly and adaptively
• Improve computational efficiency in pose-lifting tasks

Innovation

• Structure-aware State Integrator for joint topology
• Motion-adaptive State Modulator for motion dynamics
• Mamba-based 3D pose estimation with SAMA framework