Ambient Diffusion Policy: Imitation Learning from Suboptimal Data in Robotics

2026-06-10

AI Summary

This work addresses the challenge of effectively leveraging large-scale, low-quality or out-of-distribution suboptimal robotic demonstration data in imitation learning. To this end, the authors propose a diffusion model–based approach that exploits the spectral power-law characteristics inherent in action sequences. By designing a noise-dependent data utilization mechanism, the method selectively extracts useful information from suboptimal demonstrations under varying noise levels and constructs a hierarchical learning architecture progressing from global to local policy refinement. Integrating diffusion policies, spectral analysis, and heterogeneous demonstration data, the proposed framework achieves substantial performance gains across six benchmark tasks and four types of suboptimal data, yielding up to a 33% improvement over current co-training baselines on the Open X-Embodiment dataset.

Abstract

We propose Ambient Diffusion Policy, a simple and principled method for imitation learning from suboptimal data in robotics. High-quality, task-specific robot data is expensive and time-consuming to collect, while suboptimal datasets with lower-quality or out-of-distribution demonstrations are abundant. Existing methods that co-train on both data sources in robotics often fail to separate the meaningful and the harmful features in the suboptimal samples. In contrast, our method extracts only the useful features by introducing a new axis to co-training in robotics: noise-dependent data usage. Ambient Diffusion Policy restricts the contribution of suboptimal data during training to only the high and low diffusion times. To rigorously justify our approach, we first observe that robot action data exhibits a spectral power law. This induces two important properties on the optimal Diffusion Policy that we exploit: a global-to-local hierarchy and locality. We theoretically formalize this discussion using a simplified model. Our experiments validate Ambient Diffusion Policy on four types of suboptimal action data (noisy trajectories, sim-to-real gap, task mismatch, and large-scale data mixtures) across six tasks. The results show that it effectively learns from arbitrary sources of suboptimal data. Notably, it outperforms existing co-training baselines by up to 33% when scaled to Open X-Embodiment - a large dataset with heterogeneous data quality and unstructured distribution shifts. Overall, Ambient Diffusion Policy increases the utility of suboptimal demonstrations and expands the set of usable data sources in robotics.

Problem

• imitation learning
• suboptimal data
• robotics
• diffusion policy
• data heterogeneity

Innovation

• Ambient Diffusion Policy
• Imitation Learning
• Suboptimal Data
• Diffusion Models
• Robotics