Technology Category
Computer Vision: Language and VisionIntelligent Robots: Multimodal Perception & Sensor FusionHumans and AI: Human-Aware Planning and Behavior Prediction
LaST0: Latent Spatio-Temporal Chain-of-Thought for Robotic Vision-Language-Action Model
🤖 AI Summary
This work addresses the limitations of existing vision-language-action models, which rely on explicit language or visual generation during inference, resulting in high latency and insufficient representation of fine-grained physical dynamics. To overcome these challenges, the authors propose a latent spatiotemporal chain-of-thought mechanism integrated within a dual-system hybrid Transformer architecture. This framework implicitly models future visual observations, 3D scene structure, and robot proprioceptive states at a low frequency, while coordinating with high-frequency action generation to enable adaptive reasoning–action switching across heterogeneous temporal scales. By fusing multimodal perception—encompassing vision, 3D geometry, and proprioception—and employing an asynchronous training strategy, the method achieves an average success rate improvement of 8% in 10 simulated tasks and 13% in 6 real-world tasks, alongside significantly accelerated inference speed.
Technology Category
Application Category
📝 Abstract
Vision-Language-Action (VLA) models have recently shown strong generalization, with some approaches seeking to explicitly generate linguistic reasoning traces or predict future observations prior to execution. However, explicit reasoning typically incurs non-negligible inference latency, which constrains the temporal resolution required for robotic manipulation. Moreover, such reasoning is confined to the linguistic space, imposing a representational bottleneck that struggles to faithfully capture ineffable physical attributes. To mitigate these limitations, we propose LaST$_0$, a framework that enables efficient reasoning before acting through a Latent Spatio-Temporal Chain-of-Thought (CoT), capturing fine-grained physical and robotic dynamics that are often difficult to verbalize. Specifically, we introduce a token-efficient latent CoT space that models future visual dynamics, 3D structural information, and robot proprioceptive states, and further extends these representations across time to enable temporally consistent implicit reasoning trajectories. Furthermore, LaST$_0$ adopts a dual-system architecture implemented via a Mixture-of-Transformers design, where a reasoning expert conducts low-frequency latent inference and an acting expert generates high-frequency actions conditioned on robotics-oriented latent representations. To facilitate coordination, LaST$_0$ is trained with heterogeneous operation frequencies, enabling adaptive switching during deployment. Across 10 real-world tasks spanning tabletop, mobile, and dexterous hand manipulation, LaST$_0$ improves mean success rates by 13%, 14% and 14% over prior SOTA VLA methods, respectively.
Problem
Research questions and friction points this paper is trying to address.
Vision-Language-Action
reasoning latency
representational bottleneck
robotic manipulation
implicit reasoning
Innovation
Methods, ideas, or system contributions that make the work stand out.
Latent Chain-of-Thought
Vision-Language-Action Model
Spatio-Temporal Reasoning
Mixture-of-Transformers
Robotic Manipulation
🔎 Similar Papers
No similar papers found.
