Innovation
Methods, ideas, or system contributions that make the work stand out.
Negation Understanding
Vision-Language-Action Models
Parameter-Efficient Fine-Tuning
Negation Transformation Layers
Teacher-Guided Alignment
Tell Robot What Not to Do: A Negation Understanding Perspective
🤖 AI Summary
This study addresses the challenge of robots executing natural language instructions containing negation constraints by proposing NegaAlign, a framework that extends Vision-Language-Action (VLA) models as a plug-in module. NegaAlign introduces a negation transformation layer to reshape instruction representations and employs a teacher-guided alignment mechanism for parameter-efficient fine-tuning, enabling the model to follow exclusionary constraints while keeping pretrained weights frozen. Additionally, a dedicated evaluation benchmark, NegaBench, is constructed. Experimental results demonstrate that this framework increases the success rate of the π0.5 model on negated instructions from 2.6% to 88.45%, achieving 88.8% in real-world scenarios without compromising its original performance.
📝 Abstract
Instruction following enables robots to perform diverse tasks specified in natural language, making it a fundamental capability for human-robot interaction. Beyond communicating desired outcomes, users also need to specify constraints on what not to do. We investigate how to enable vision-language-action models (VLAs) to follow negated instructions, where robots must accomplish task goals while respecting explicit exclusions. To this end, we propose NegaAlign, a parameter-efficient, plug-and-play framework that extends pretrained VLAs to follow negated instructions through image-language supervision alone. Specifically, we introduce Negation Transformation Layers into selected layers of the vision-language backbone to reshape intermediate instruction representations. Meanwhile, a teacher-guided alignment mechanism is designed to align instruction-relevant visual tokens, transferring action-relevant grounding from instructions that satisfy the negated constraint. The training phase uses supervision constructed from existing demonstrations and updates only the inserted layers, keeping all pretrained parameters frozen, including the action generator. We further introduce NegaBench, a simulation benchmark spanning 10 scenarios across five domains for systematically evaluating manipulation under negated constraints. Experiments across GR00T, $π_0$, and $π_{0.5}$ demonstrate consistent improvements in negated instruction following. With 11.6M trainable parameters, NegaAlign increases the negated-instruction success rate of $π_{0.5}$ from 2.60% to 88.45% on NegaBench and from 12.4% to 88.8% on real-world tasks, while retaining performance on affirmative instructions.
🔎 Similar Papers
2024-09-09arXiv.orgCitations: 2
