behavior cloning

This work addresses the practical bottleneck in imitation learning—scarce access to expert action labels—by systematically studying “Learning from Observation” (LfO), which relies solely on expert state sequences. We propose the first unified taxonomy for LfO, organizing methods along two orthogonal dimensions: modeling objectives (state reconstruction, latent variable inference, policy alignment) and algorithmic mechanisms (integration with RL, model-based prediction, or hierarchical architectures). We establish, for the first time, rigorous theoretical connections between LfO and offline RL, model-based RL, and hierarchical RL. Our analysis precisely characterizes key assumptions, fundamental performance limits, and domain-specific applicability conditions for each method class. Furthermore, we identify major open challenges—including ambiguity in inverse dynamics and distributional shift—and propose empirically verifiable pathways toward resolution. This work lays a structured conceptual foundation for LfO, advancing its development toward greater robustness, interpretability, and real-world applicability.

Existing imitation learning methods typically rely on action labels, assume a unique optimal action for each state, and overlook discrepancies in environmental dynamics, making it challenging to learn effective policies from observed trajectories in a fully unsupervised manner. This work proposes UfO, the first entirely unsupervised imitation learning framework, which operates via a two-stage process: first, it infers the teacher’s implicit actions from state transitions through conditional transition estimation; second, it dynamically aligns the agent’s trajectory with the teacher’s behavior using an online behavioral alignment mechanism. By eliminating reliance on action supervision and the single-action assumption, UfO consistently outperforms both the teacher policy and other ILfO approaches across five benchmark environments, achieving the lowest standard deviation and demonstrating superior generalization and stability.

This paper identifies a fundamental error-amplification problem in imitation learning for continuous state-action spaces: even under stable system dynamics and smooth, deterministic expert policies, any smooth deterministic imitator incurs execution errors that grow exponentially with task horizon—a theoretical bottleneck pervasive in behavioral cloning and offline reinforcement learning. The authors provide the first rigorous proof of this exponential amplification in continuous-action settings, and identify viable mitigation strategies: adopting nonsmooth, non-Markovian, or highly stochastic policies, or leveraging expert datasets with sufficient action-space dispersion. Leveraging contraction theory and control-theoretic analysis, the paper further demonstrates that parametrization techniques—such as action chunking and diffusion-based policy representations—significantly suppress error accumulation. These results establish critical theoretical limits for robotic imitation learning and yield novel design principles for robust policy learning in continuous domains.

This work addresses offline imitation learning from imperfect human demonstrations—characterized by noisy actions and suboptimal policies—by proposing Counterfactual Behavior Cloning (Counter-BC). The method explicitly models and corrects demonstration bias through counterfactual action generation, producing semantically consistent augmentations that disentangle and recover the demonstrator’s underlying intent policy from superficial behavioral traces. Built upon the behavior cloning framework, Counter-BC integrates counterfactual action sampling, consistency regularization, and a theoretically grounded policy disentanglement mechanism to enable robust optimization in the offline setting. Experiments on both simulated and real-world robotic platforms demonstrate that Counter-BC significantly outperforms existing baselines. It reliably recovers concise, consistent, and generalizable policies from highly noisy, multi-user, and low-skill demonstrations—without requiring online interaction or reward labels.

Imitation learning suffers from two interrelated challenges: error accumulation and train-deployment distributional shift, leading to poor generalization. To address these, we propose a multi-step state prediction framework inspired by model predictive control (MPC), establishing the first model-based imitation learning paradigm that unifies multi-step forward dynamics modeling with behavior cloning. Our method explicitly models system dynamics and jointly optimizes multi-step state predictions and action outputs. We theoretically derive upper bounds on sample complexity and policy error convergence. Empirically, evaluated on standard benchmarks, our approach significantly outperforms conventional behavior cloning. It demonstrates superior robustness to both distributional shift and observation noise, validating its practical effectiveness while maintaining theoretical rigor.

This work addresses the high cost of state-level annotation and the performance limitations of behavioral cloning (BC) in imitation learning. To this end, we propose a hybrid paradigm integrating offline demonstrations with interactive state labeling. Theoretically, we provide the first rigorous proof that interactive state labeling achieves strictly superior performance per unit annotation cost compared to BC. Methodologically, we introduce Warm Stagger—a novel algorithm that jointly optimizes cold-start initialization and cumulative error propagation—leveraging a DAgger variant for interactive learning and establishing convergence guarantees and sample efficiency via MDP-theoretic analysis. Empirically, on MuJoCo continuous-control benchmarks, our approach consistently outperforms BC under moderate annotation budgets and attains learning efficiency approaching that of an optimal single-data-source oracle. Our core contributions are: (i) establishing the theoretical advantage of interactive annotation over passive BC, and (ii) introducing the first practical, provably efficient hybrid imitation learning framework.

Imitation learning is a popular paradigm to teach robots new tasks, but collecting robot demonstrations through teleoperation or kinesthetic teaching is tedious and time-consuming. In contrast, directly demonstrating a task using our human embodiment is much easier and data is available in abundance, yet transfer to the robot can be non-trivial. In this work, we propose Real2Gen to train a manipulation policy from a single human demonstration. Real2Gen extracts required information from the demonstration and transfers it to a simulation environment, where a programmable expert agent can demonstrate the task arbitrarily many times, generating an unlimited amount of data to train a flow matching policy. We evaluate Real2Gen on human demonstrations from three different real-world tasks and compare it to a recent baseline. Real2Gen shows an average increase in the success rate of 26.6% and better generalization of the trained policy due to the abundance and diversity of training data. We further deploy our purely simulation-trained policy zero-shot in the real world. We make the data, code, and trained models publicly available at real2gen.cs.uni-freiburg.de.

This work addresses safety-constrained imitation learning, aiming to learn a maximum-entropy policy from expert demonstrations while satisfying multiple constraints (e.g., safety requirements and task specifications). We propose a novel probabilistic inference–based imitation learning framework that formulates constraint satisfaction as a KL-divergence–constrained optimization problem, yielding a unified objective accommodating both hard and soft constraints. To jointly optimize policy entropy and constraint violation cost, we introduce a dual gradient descent algorithm with theoretical convergence guarantees. Theoretically, we establish a rigorous equivalence between constrained imitation learning and constrained maximum-entropy reinforcement learning. Empirically, our method achieves significant improvements over state-of-the-art baselines across diverse simulated and real-world robotic tasks, demonstrating superior generalization, behavioral diversity, and robustness to constraint violations.

To address the challenges of absent action labels, reward signals, and limited interactivity in video-based imitation learning, this paper proposes a purely video-driven, high-sample-efficiency visual policy learning framework. Methodologically, it first employs self-supervised learning to extract action-relevant latent representations; second, it constructs a dynamics-driven, unsupervised inter-frame latent action prediction model; and third, it performs online alignment of latent actions to the real action space, enabling joint iterative refinement of latent actions and policy cloning. This work provides the first theoretical and empirical demonstration that efficient visual policy learning is feasible using only raw demonstration videos—without expert action annotations or reward signals. Evaluated on 28 visual control tasks, the method outperforms state-of-the-art video imitation learning and reward-based reinforcement learning approaches in sample efficiency on 24 tasks, achieving expert-level performance in several.

This work proposes a semi-supervised imitation learning approach based on an inverse dynamics model (IDM) in settings with limited action-labeled trajectories and abundant unlabeled trajectories. The IDM predicts actions from state transitions and can either serve directly as a policy (VM-IDM) or generate pseudo-labels for unlabeled data (IDM labeling). Theoretical analysis reveals that IDM outperforms behavioral cloning primarily due to its lower hypothesis class complexity and reduced stochasticity, leading to higher sample efficiency. Building on this insight, the authors enhance the LAPO algorithm and validate the proposed method within a unified video-action prediction (UVA) framework. Both theoretical analysis and empirical results consistently demonstrate the superior sample efficiency of the IDM-based approach.

To address the bottleneck in multi-agent imitation learning—namely, reliance on synchronized joint demonstrations—this paper proposes Round-Robin Behavior Cloning (RR-BC). RR-BC enables a single human operator to collect asynchronous demonstration data by temporally alternating control over agents one at a time, eliminating the need for joint-action space annotation. Built upon the behavior cloning framework, it integrates serialized demonstration modeling with decentralized policy training, enabling cooperative policy learning while respecting individual observation constraints. Evaluated on four simulated tasks, RR-BC matches or surpasses synchronous-demonstration baselines; it further succeeds in two real-robot collaborative tasks, demonstrating strong generalization and practical applicability. The core contribution is the first systematic solution to the challenge of learning from asynchronous, single-operator multi-agent demonstrations—significantly lowering the data collection barrier for multi-agent imitation learning.