PAMI: Part Anchored Motion for Text to Human-Object Interaction Generation

2026-09-29

AI Summary

This study addresses object drifting, contact loss, and penetration in text-driven human-object interaction generation caused by implicit global coupling. To this end, we propose the PAMI framework. Inspired by the Hough transform, PAMI introduces a multi-body-part anchor voting mechanism to localize object motion and constructs a structured interaction latent space via PamiVAE. Furthermore, it employs a coarse-to-fine hierarchical strategy that integrates PamiGen with PamiRefiner—equipped with long-range probes and short-range sensors—to recursively resolve fine-grained contact geometry. Extensive evaluations on the InterAct dataset demonstrate that the proposed method substantially enhances interaction realism and motion accuracy, achieving a 14.5% improvement in contact recall over state-of-the-art approaches.

Abstract

Text-conditioned full-body human-object interaction (HOI) generation requires synthesizing human motion and object trajectories that match the input text while remaining precisely coordinated over time. Most methods represent the human and object as separate trajectories and predict the global human-object couplings. Learning this complex, dynamically changing relationship implicitly, however, often yields object drift, missed contact, and penetration. We introduce PAMI, a Part-Anchored Motion framework for Interaction generation. Inspired by the classic Hough Transform, our key idea is to localize object motion by letting body-part anchors vote for it: we express object motion relative to multiple body-part anchors and use PamiVAE to learn an interaction latent space, decoding frame-wise weights that aggregate these part-specific votes. Building on this representation, PAMI generates interactions in a coarse-to-fine hierarchy. PamiGen first generates a coarse human-object interaction from text in this structured latent space, and PamiRefiner then recursively resolves fine-grained contact geometry using a hybrid surface-sensing representation, combining long-range probes that capture overall body-part influence with short-range sensors that resolve detailed contacts near the object surface. Experiments on InterAct show that PAMI generates more faithful interactions and more accurate human-relative object motion than previous methods, achieving 14.5% higher contact recall than the previous state of the art. Extensive ablations validate the contributions of both the part-anchored voting representation and hybrid surface-sensing refinement.

Problem

• Text-to-HOI generation
• Human-object interaction
• Object drift
• Contact loss
• Penetration

Innovation

• Part-Anchored Voting
• Human-Object Interaction Generation
• Coarse-to-Fine Hierarchy
• Hybrid Surface-Sensing
• PamiVAE