AgenticTactileVLA: Contact-Guided Execution-Time Supervision for Generalizable Dexterous Manipulation without VLA Retraining

2026-10-03

AI Summary

This study addresses the limited robustness of Vision-Language-Action (VLA) models in dexterous manipulation under geometric variations and occlusions by proposing a real-time, retraining-free proprioceptive supervision mechanism. This approach defers object adaptation from the prediction stage to the physical interaction phase, introducing a novel execution-time contact guidance paradigm without tactile sensors. By leveraging finger position and motor torque feedback, the method dynamically adjusts finger flexion-extension and control modes. Experiments conducted on the Unitree G1 platform equipped with a Revo2 hand demonstrate that the task success rate for novel objects improves to 84.0%, while correction triggers are reduced by 65.7%. These results indicate that the proposed mechanism significantly enhances the generalization capability of fixed VLA models to unseen objects.

Abstract

Vision-language-action policies may predict a transferable manipulation strategy yet fail to realize it reliably on the encountered object: objects compatible with the same grasp differ in geometry and compliance, and visual feedback degrades under closure occlusion. AgenticTactileVLA is presented as an execution-time supervisor that shifts part of object-specific adaptation from prediction to physical interaction. A fixed VLA provides the approach and hand targets; the supervisor decides whether to remain transparent, refine finger flexion, retain or release the corrected configuration, return control to the VLA for retry, or select a compliant hand-control regime. It uses finger-position and motor-effort feedback as proprioceptive contact evidence and requires neither tactile sensors nor VLA retraining. On a Unitree G1 with a BrainCo Revo2 hand, a randomized matched-block evaluation on five objects held out from VLA training yields 61.3% completion for the base VLA, 72.0% for unconditional close-to-stall control, and 84.0% for the supervisor under a shared budget; the gain is positive on every object and persists under moderate pose perturbations. Ablations show the gain is not explained by extended closure alone, and that selective triggering reduces correction episodes by 65.7% with no detected change in completion. A retention audit shows acceptance predicts retention in 88.9% of held-out cases, while compliant objects expose conservative false rejection. A thin-walled-cup study demonstrates contextual routing to compliant control, matching an always-compliant reference. These results suggest that contact-guided execution-time adaptation can improve the object-level generalization of a fixed VLA to held-out objects by adapting physical realization without object-specific retraining.

Problem

• Vision-Language-Action (VLA)
• Dexterous Manipulation
• Occlusion
• Object Generalization
• Contact Feedback

Innovation

• execution-time supervision
• dexterous manipulation
• proprioceptive contact
• VLA generalization
• compliant control