Noise-Guided Transport for Imitation Learning

2025-09-30

AI Summary

This work addresses imitation learning under extremely low-data regimes—only 20 expert trajectories. We propose NGT, a lightweight off-policy method that formulates imitation as a noise-guided optimal transport problem. NGT implicitly aligns the expert and agent policy distributions via adversarial training, eliminating the need for pretraining, large-scale models, or specialized architectures. Its core innovation is a controllable noise mechanism integrated into the optimal transport framework, enabling intrinsic uncertainty estimation and robust policy learning. NGT is optimized efficiently using only small neural networks, features straightforward hyperparameter tuning, and enables rapid deployment. On continuous-control benchmarks—including high-dimensional Humanoid—NGT significantly outperforms existing low-data imitation learning approaches. The implementation is publicly available.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Search and Optimization: Learning to Search
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

We consider imitation learning in the low-data regime, where only a limited number of expert demonstrations are available. In this setting, methods that rely on large-scale pretraining or high-capacity architectures can be difficult to apply, and efficiency with respect to demonstration data becomes critical. We introduce Noise-Guided Transport (NGT), a lightweight off-policy method that casts imitation as an optimal transport problem solved via adversarial training. NGT requires no pretraining or specialized architectures, incorporates uncertainty estimation by design, and is easy to implement and tune. Despite its simplicity, NGT achieves strong performance on challenging continuous control tasks, including high-dimensional Humanoid tasks, under ultra-low data regimes with as few as 20 transitions. Code is publicly available at: https://github.com/lionelblonde/ngt-pytorch.

Problem

• Addresses imitation learning with limited expert demonstrations
• Solves optimal transport problem via adversarial training method
• Achieves performance in low-data regimes with minimal transitions

Innovation

• Noise-Guided Transport solves imitation via optimal transport
• Adversarial training enables lightweight off-policy learning
• Method requires no pretraining and handles uncertainty estimation