Good in Bad (GiB): Sifting Through End-user Demonstrations for Learning a Better Policy

2026-05-02

AI Summary

This work addresses the challenge of learning from imperfect demonstrations that often contain unintentional errors, which can compromise policy safety if used directly, yet discarding them entirely wastes valuable data. To resolve this trade-off, the authors propose GiB (Granular filtering by subtask), the first algorithm capable of automatically identifying and removing low-quality segments at the subtask level while preserving high-quality portions for policy training. GiB integrates self-supervised feature extraction, binary weight annotation, modeling of high-quality segment distributions, and Mahalanobis distance-based detection to enable fine-grained filtering of mixed-quality demonstrations. Evaluated on multi-step tasks in both simulated and real-world Franka robot environments, policies trained on GiB-filtered data achieve significantly improved performance, demonstrating enhanced learning efficiency and safety—particularly in low-data regimes.

Abstract

Imitation learning offers a promising framework for enabling robots to acquire diverse skills from human users. However, most imitation learning algorithms assume access to high-quality demonstrations an unrealistic expectation when collecting data from non-expert users, whose demonstrations often contain inadvertent errors. Naively learning from such demonstrations can result in unsafe policy behavior, while discarding entire demonstrations due to occasional mistakes wastes valuable data, especially in low-data settings. In this work, we introduce GiB (Good-in-Bad), an algorithm that automatically identifies and discards erroneous subtasks within demonstrations while preserving high-quality subtasks. The filtered data can then be used by any policy learning algorithm to train more robust policies. GiB first trains a self-supervised model to learn latent features and assigns binary weights to label each demonstration as good or bad. It then models the latent feature distribution of high-quality segments and uses the Mahalanobis distance to detect and evaluate poor-quality subtasks. We validate GiB on the Franka robot in both simulated and real-world multi-step tasks, demonstrating improved policy performance when learning from mixed-quality human demonstrations.

Problem

• imitation learning
• non-expert demonstrations
• erroneous subtasks
• data quality
• policy learning

Innovation

• Imitation Learning
• Demonstration Filtering
• Self-supervised Learning
• Mahalanobis Distance
• Robotic Policy Learning