Does Continual Imitation Learning Remain Grounded? A Language-Perturbed Benchmark for Robotic Task Retention

2026-09-30

AI Summary

This study addresses the issue in continual imitation learning where robots, while retaining task skills, tend to lose their dependence on language instructions, resulting in behaviors that are not genuinely grounded in language. To investigate this, the authors construct a language perturbation benchmark based on LIBERO, generating instruction variants that either preserve or alter semantics. Furthermore, they propose a diagnostic protocol that decouples task competence from language sensitivity, serving as a complement to traditional forgetting metrics. The key contribution of this work lies in revealing that strong continual learning performance does not necessarily guarantee reliable language grounding. It also demonstrates that the proposed diagnostic tool can effectively quantify semantic robustness and identify whether acquired skills remain properly guided by language instructions.

Abstract

Continual imitation learning evaluates whether a robot can learn new knowledge without forgetting previously learned skills. However, retaining task performance does not ensure the behavior remains grounded in language because policies may rely on scene cues, object associations, or memorized task structure. We introduce a benchmark protocol to study how language-guided behavior changes as robotic policies learn successive tasks. We construct meaning-preserving and meaning-changing instruction variants for the Goal, Spatial, Object, and Long suites of LIBERO. Policy experiments focus on LIBERO-Goal, evaluating Original and Paraphrase instructions after each continual-learning stage. We compare representative continual imitation learning methods under their original assumptions while separating task competence from language sensitivity. The proposed diagnostics complement standard learning and forgetting metrics by measuring semantic robustness, goal adaptation, and language sensitivity. Results show that strong continual-learning performance does not always translate to reliable language grounding, and our diagnostics help determine whether retained skills remain correctly guided by their instructions. Additional materials are available at https://sites.google.com/view/stillgrounded

Problem

• Continual Imitation Learning
• Language Grounding
• Robotic Task Retention
• Semantic Robustness

Innovation

• Continual Imitation Learning
• Language Grounding
• Benchmark Protocol
• Semantic Robustness
• Language Perturbation