Grounded Semantic Re-Binding for Robust Instruction Generalization in Vision-Language-Action Models

2026-08-03

AI Summary

This work addresses the poor robustness of existing vision-language-action (VLA) models to minor rephrasings of instructions, demonstrating that this sensitivity stems not from inadequate semantic understanding but from the entangled encoding of task semantics and visual features. To resolve this, the authors propose a semantic rebinding mechanism that decouples semantic and visual representations, extracts them independently, and then explicitly fuses them to train a lightweight action expert module from scratch, yielding the ParaVLA model. Without relying on large-scale data, this approach substantially enhances generalization, achieving up to a 44.6% absolute improvement in success rate on the LIBERO-Para benchmark and a PRIDE score of 70.4—matching the performance of much larger baseline models.

Abstract

Vision-Language-Action (VLA) models excel in robotic manipulation but suffer catastrophic performance drops when canonical instructions are simply paraphrased. Although this brittleness is typically addressed through costly data scaling, our probing reveals that the root cause is architectural rather than a lack of semantic understanding. Specifically, we demonstrate that current VLAs successfully retain the correct task identity internally. The failure actually stems from the joint encoding of dynamic visual observations and text, which introduces systematic feature shifts. Because the downstream action policy is highly vulnerable to these variations, it fails to translate the preserved semantics into correct control commands. To resolve this structural bottleneck, we propose Grounded Semantic Re-binding (GSR), an elegant intervention that bypasses unstable joint routing by explicitly fusing independently extracted task semantics with native visual features to train a completely re-initialized action expert from scratch. This targeted intervention dramatically restores paraphrastic invariance using only canonical demonstrations. On the LIBERO-Para benchmark, GSR improves success rates by up to 44.6 percent. It enables lightweight models to rival massively scaled baselines and pushes state-of-the-art models to a new record PRIDE score of 70.4, outperforming the recently introduced large-scale pretrained model Xiaomi-Robotics-0 in instruction generation capabilities. Building on these insights, we also introduce ParaVLA, a natively decoupled 0.33B-parameter model exhibiting near-perfect robustness to instruction rewording. Ultimately, our work proves that robust semantic grounding can be achieved through elegant structural design, bypassing the inefficient brute-force data scaling paradigm.

Problem

• instruction generalization
• vision-language-action models
• paraphrastic invariance
• semantic grounding
• feature shift

Innovation

• Grounded Semantic Re-binding
• Vision-Language-Action models
• instruction generalization
• feature decoupling
• paraphrastic invariance