Videos are Sample-Efficient Supervisions: Behavior Cloning from Videos via Latent Representations

2025-12-25

AI Summary

To address the challenges of absent action labels, reward signals, and limited interactivity in video-based imitation learning, this paper proposes a purely video-driven, high-sample-efficiency visual policy learning framework. Methodologically, it first employs self-supervised learning to extract action-relevant latent representations; second, it constructs a dynamics-driven, unsupervised inter-frame latent action prediction model; and third, it performs online alignment of latent actions to the real action space, enabling joint iterative refinement of latent actions and policy cloning. This work provides the first theoretical and empirical demonstration that efficient visual policy learning is feasible using only raw demonstration videos—without expert action annotations or reward signals. Evaluated on 28 visual control tasks, the method outperforms state-of-the-art video imitation learning and reward-based reinforcement learning approaches in sample efficiency on 24 tasks, achieving expert-level performance in several.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Computer Vision: Video Understanding & Activity Analysis
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Humans can efficiently extract knowledge and learn skills from the videos within only a few trials and errors. However, it poses a big challenge to replicate this learning process for autonomous agents, due to the complexity of visual input, the absence of action or reward signals, and the limitations of interaction steps. In this paper, we propose a novel, unsupervised, and sample-efficient framework to achieve imitation learning from videos (ILV), named Behavior Cloning from Videos via Latent Representations (BCV-LR). BCV-LR extracts action-related latent features from high-dimensional video inputs through self-supervised tasks, and then leverages a dynamics-based unsupervised objective to predict latent actions between consecutive frames. The pre-trained latent actions are fine-tuned and efficiently aligned to the real action space online (with collected interactions) for policy behavior cloning. The cloned policy in turn enriches the agent experience for further latent action finetuning, resulting in an iterative policy improvement that is highly sample-efficient. We conduct extensive experiments on a set of challenging visual tasks, including both discrete control and continuous control. BCV-LR enables effective (even expert-level on some tasks) policy performance with only a few interactions, surpassing state-of-the-art ILV baselines and reinforcement learning methods (provided with environmental rewards) in terms of sample efficiency across 24/28 tasks. To the best of our knowledge, this work for the first time demonstrates that videos can support extremely sample-efficient visual policy learning, without the need to access any other expert supervision.

Problem

• Enables imitation learning from videos without action or reward signals
• Extracts latent actions from videos for sample-efficient policy learning
• Improves visual policy performance with minimal environmental interactions

Innovation

• Self-supervised latent action extraction from videos
• Dynamics-based unsupervised objective for action prediction
• Iterative online fine-tuning with minimal real interactions