Explicit Kinematic Guidance from Analytic Concepts for Vision-Language-Action Models

2026-07-29

AI Summary

Current vision–language–action (VLA) models predominantly rely on 2D inputs, limiting their ability to capture 3D object structure and commonsense knowledge, thereby constraining spatial reasoning and fine-grained manipulation. This work proposes the Concept Expert module, which introduces explicit, executable procedural concepts as intermediate representations for the first time. Prior to inference, a foundation vision model extracts 3D kinematic and structural parameters of objects, which are dynamically tracked and updated during interaction to provide dense rewards and precise spatial guidance for the VLA policy. This approach seamlessly integrates 3D structural awareness with end-to-end VLA learning, significantly improving task success rates and sample efficiency under both supervised and reinforcement learning settings, thereby demonstrating the critical role of structured conceptual guidance in enhancing the accuracy and generalization of physical interaction.

Abstract

Current Vision-Language-Action (VLA) models rely mainly on 2D inputs, neglecting the rich object structural information and commonsense knowledge inherent in the 3D physical world. This deficiency restricts their spatial awareness and adaptability for complex, high-precision manipulation. To bridge this crucial gap, we construct a Concept Expert module for VLA to build executable Analytic Concepts that represent objects as explicit, programmatic blueprints. Our mechanism operates in two synergistic phases: First, prior to VLA inference, the Concept Expert leverages 3D information from Vision Foundation Models (VFMs) to estimate the initial kinematic and structural parameters. Second, throughout the manipulation process, the VLA model utilizes its inherent capability to dynamically track the dynamic concept parameters, continuously aligning them with observational changes to ensure persistent accuracy. Once established, the Analytic Concepts provide explicit, high-quality guidance for VLA fine-tuning through (1) dense, programmatic manipulation rewards and (2) precise spatial guidance. This formulation allows VLA models to learn physically grounded interaction behaviors while maintaining end-to-end learning flexibility. Our experimental results show consistent improvements in success rate and learning efficiency across supervised and reinforcement learning settings, demonstrating the effectiveness of structured, concept-based guidance for VLA post-training.

Problem

• Vision-Language-Action models
• 3D structural information
• spatial awareness
• high-precision manipulation
• commonsense knowledge

Innovation

• Analytic Concepts
• Vision-Language-Action Models
• 3D Kinematic Guidance
• Concept Expert
• Programmatic Manipulation Rewards