Closing the Loop in Humanoid VLA: Persistent 3D Object Tokens for Verifiable Loco-Manipulation

2026-07-20

AI Summary

This work addresses the challenge of long-horizon mobile manipulation in humanoid robots, where inconsistent object states—caused by motion, contact, and occlusion—lead to a disconnect between action execution and verification. The authors propose Persistent Object Tokenization (POT), a novel approach that introduces role-indexed 3D object-centric states derived from RGB-D observations as a unified abstraction. By tokenizing objects, POT enables visual–language–action policies and geometric predicate verification to operate on a shared, executable, and verifiable representation of object states, thereby closing the perception–action loop. Evaluated on the Unitree G1 platform, the method improves task success from 39/80 to 71/80 and achieves 44/50 on the Being-0 service benchmark (up from 37/50), with particularly notable gains in tasks requiring sustained 3D spatial relationships.

Abstract

Vision-language-action policies are a promising foundation for general robot control, but long-horizon humanoid loco-manipulation requires the robot to treat task objects as persistent physical entities across movement, contact, occlusion, and recovery. We study this problem as object-state divergence: the object state used to condition a whole-body action can differ from the state used to decide whether the action achieved the intended physical relation. We propose \emph{Persistent Object Tokenization} (POT), which maintains role-indexed 3D object records from RGB-D observations and converts them into object tokens for a whole-body action expert. Instantiated as \emph{POT-VLA}, the same object records condition action generation and support geometric predicate checks, yielding a closed-loop execution system in which object state is both actionable and verifiable. On a Unitree G1, POT-VLA improves a matched direct GR00T-N1.7 baseline from 39/80 to 71/80 successes over eight real-world task families. In an external Being-0-aligned reference, POT-VLA achieves 44/50 successes on aligned service tasks, compared with the 37/50 success reported by the Being-0 paper. The largest gains occur on tasks requiring maintained 3D relations, suggesting that persistent object-centered state is a useful abstraction for verifiable humanoid VLA execution.

Problem

• humanoid loco-manipulation
• object-state divergence
• persistent object representation
• vision-language-action
• 3D object tokens

Innovation

• Persistent Object Tokenization
• Vision-Language-Action
• 3D Object Tokens
• Closed-loop Execution
• Humanoid Loco-Manipulation