Spatiotemporal-Untrammelled Mixture of Experts for Multi-Person Motion Prediction

2025-12-25

AI Summary

To address the rigidity in spatiotemporal dependency modeling—constrained by positional encoding—and the high computational cost—due to the quadratic complexity of standard attention—in multi-person motion prediction, this paper proposes a positional-encoding-free, low-complexity Mixture-of-Experts (MoE) architecture. Methodologically, it introduces four heterogeneous spatiotemporal experts operating under a novel collaborative mechanism; employs bidirectional spatiotemporal Mamba as a lightweight expert to jointly enable parameter sharing and dynamic sparse routing; and adopts spatiotemporal decoupled modeling with position-free sequence representation. Evaluated on four benchmark datasets, the method achieves state-of-the-art accuracy while reducing model parameters by 41.38% and accelerating training by 3.6× compared to prior approaches.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Computer Vision: Motion & Tracking
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

Comprehensively and flexibly capturing the complex spatio-temporal dependencies of human motion is critical for multi-person motion prediction. Existing methods grapple with two primary limitations: i) Inflexible spatiotemporal representation due to reliance on positional encodings for capturing spatiotemporal information. ii) High computational costs stemming from the quadratic time complexity of conventional attention mechanisms. To overcome these limitations, we propose the Spatiotemporal-Untrammelled Mixture of Experts (ST-MoE), which flexibly explores complex spatio-temporal dependencies in human motion and significantly reduces computational cost. To adaptively mine complex spatio-temporal patterns from human motion, our model incorporates four distinct types of spatiotemporal experts, each specializing in capturing different spatial or temporal dependencies. To reduce the potential computational overhead while integrating multiple experts, we introduce bidirectional spatiotemporal Mamba as experts, each sharing bidirectional temporal and spatial Mamba in distinct combinations to achieve model efficiency and parameter economy. Extensive experiments on four multi-person benchmark datasets demonstrate that our approach not only outperforms state-of-art in accuracy but also reduces model parameter by 41.38% and achieves a 3.6x speedup in training. The code is available at https://github.com/alanyz106/ST-MoE.

Problem

• Captures complex spatio-temporal dependencies in human motion
• Reduces high computational costs of existing methods
• Improves accuracy while decreasing model parameters and training time

Innovation

• Uses bidirectional spatiotemporal Mamba as experts
• Incorporates four distinct spatiotemporal experts
• Reduces computational cost and model parameters