SemGeoMo: Dynamic Contextual Human Motion Generation with Semantic and Geometric Guidance

2025-03-03

AI Summary

To address insufficient semantic plausibility and geometric accuracy in human interaction motion generation within dynamic environments, this paper proposes a diffusion-based generative framework jointly guided by text, functional, and joint-level semantics and geometry. Methodologically, it integrates CLIP-based textual encoding, functional graph modeling, and skeletal geometric constraint losses to enable multimodal conditional generation. Crucially, we introduce a novel cross-level semantic alignment mechanism that simultaneously ensures task-intent consistency and physical feasibility throughout motion synthesis. Evaluated on three standard benchmarks, our approach achieves state-of-the-art performance, demonstrating significantly improved generalization to unseen interactive objects and scene layouts. Quantitative and qualitative results confirm superior motion quality and environmental adaptability compared to existing methods.

Technology Category

• Natural Language Processing: Generation
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Computer Vision: Diffusion Models for Vision

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Generating reasonable and high-quality human interactive motions in a given dynamic environment is crucial for understanding, modeling, transferring, and applying human behaviors to both virtual and physical robots. In this paper, we introduce an effective method, SemGeoMo, for dynamic contextual human motion generation, which fully leverages the text-affordance-joint multi-level semantic and geometric guidance in the generation process, improving the semantic rationality and geometric correctness of generative motions. Our method achieves state-of-the-art performance on three datasets and demonstrates superior generalization capability for diverse interaction scenarios. The project page and code can be found at https://4dvlab.github.io/project_page/semgeomo/.

Problem

• Generates human interactive motions in dynamic environments.
• Improves semantic rationality and geometric correctness of motions.
• Achieves state-of-the-art performance across diverse interaction scenarios.

Innovation

• Leverages text-affordance-joint multi-level guidance
• Improves semantic rationality and geometric correctness
• Achieves state-of-the-art performance on datasets