Stochastic Human Motion Prediction with Memory of Action Transition and Action Characteristic

2025-07-05

AI Summary

Predicting future human motion for a target action from a non-target action history sequence faces two key challenges: (1) discontinuous inter-action transitions due to large velocity disparities between actions, and (2) feature ambiguity and prediction distortion arising from semantic similarity among distinct actions. To address these, we propose an action-driven stochastic human motion prediction framework. Our core contributions are: (1) a soft-transition action library and an action feature library that jointly model inter-action transition dynamics and discriminative action representations; and (2) a memory bank with a soft-search mechanism, integrated via adaptive attention fusion to dynamically calibrate action context and target-oriented features. Extensive experiments on four mainstream motion prediction benchmarks demonstrate significant improvements over state-of-the-art methods, yielding more natural, temporally coherent, and semantically accurate motion sequences. Code and interactive demos are publicly available.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Computer Vision: Motion & Tracking
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Action-driven stochastic human motion prediction aims to generate future motion sequences of a pre-defined target action based on given past observed sequences performing non-target actions. This task primarily presents two challenges. Firstly, generating smooth transition motions is hard due to the varying transition speeds of different actions. Secondly, the action characteristic is difficult to be learned because of the similarity of some actions. These issues cause the predicted results to be unreasonable and inconsistent. As a result, we propose two memory banks, the Soft-transition Action Bank (STAB) and Action Characteristic Bank (ACB), to tackle the problems above. The STAB stores the action transition information. It is equipped with the novel soft searching approach, which encourages the model to focus on multiple possible action categories of observed motions. The ACB records action characteristic, which produces more prior information for predicting certain actions. To fuse the features retrieved from the two banks better, we further propose the Adaptive Attention Adjustment (AAA) strategy. Extensive experiments on four motion prediction datasets demonstrate that our approach consistently outperforms the previous state-of-the-art. The demo and code are available at https://hyqlat.github.io/STABACB.github.io/.

Problem

• Generating smooth transitions between varying action speeds
• Learning distinct characteristics of similar actions
• Ensuring consistent and reasonable motion prediction results

Innovation

• Soft-transition Action Bank stores action transition information
• Action Characteristic Bank records action-specific prior information
• Adaptive Attention Adjustment fuses features from both banks