environment interaction

This work addresses the practical bottleneck in imitation learning—scarce access to expert action labels—by systematically studying “Learning from Observation” (LfO), which relies solely on expert state sequences. We propose the first unified taxonomy for LfO, organizing methods along two orthogonal dimensions: modeling objectives (state reconstruction, latent variable inference, policy alignment) and algorithmic mechanisms (integration with RL, model-based prediction, or hierarchical architectures). We establish, for the first time, rigorous theoretical connections between LfO and offline RL, model-based RL, and hierarchical RL. Our analysis precisely characterizes key assumptions, fundamental performance limits, and domain-specific applicability conditions for each method class. Furthermore, we identify major open challenges—including ambiguity in inverse dynamics and distributional shift—and propose empirically verifiable pathways toward resolution. This work lays a structured conceptual foundation for LfO, advancing its development toward greater robustness, interpretability, and real-world applicability.

Current AI agents face significant challenges in unifying cognitive modeling, planning, and interactive behavior, as well as ensuring reliable deployment. This paper proposes a unified agent framework that integrates principles from cognitive science, hierarchical reinforcement learning (HRL), and large language model (LLM)-based reasoning to systematically unify perception, decision-making, and interaction. Methodologically, it employs interdisciplinary collaborative modeling, incorporates explainability mechanisms and formal safety constraints, and synergizes multi-agent coordination with deep reinforcement learning to enhance robustness and adaptivity in dynamic, complex environments. The core contributions are threefold: (1) the first end-to-end theoretical pathway bridging cognitive modeling to trustworthy deployment; (2) identification of key technical breakthrough directions; and (3) an architecture blueprint and practical implementation guidelines for next-generation trustworthy, adaptive intelligent systems—balancing theoretical rigor with engineering feasibility.

Web agents exhibit poor adaptability to novel websites, while real-world task data is scarce and existing synthetic data suffers from hallucination, redundancy, and misaligned execution trajectories. Method: We propose a two-stage refinement framework that jointly optimizes synthetic task validity and trajectory fidelity. Our approach integrates webpage-element-classification-guided task generation, runtime task correction, and global-context-aware trajectory refinement to construct high-quality synthetic supervision signals, followed by fine-tuning of open-source web agents. Contribution/Results: Experiments demonstrate substantial improvements in task success rates across multiple unseen websites, outperforming state-of-the-art synthetic data methods. Our framework establishes a scalable, high-fidelity paradigm for synthetic data construction, enhancing web agent generalization under low-resource conditions.

Existing approaches fail to address planning failures caused by emergent novelties in open-world environments, lacking support for continuous spatiotemporal scheduling, multi-agent coordination, and memory-augmented adaptation, while relying on low-fidelity validation environments. Method: This paper proposes a general autonomous agent framework integrating symbol–neural hybrid modeling, online model refinement, distributed temporal scheduling, and memory-enhanced metacognitive control—forming a closed-loop “scheduler–executor–memory” architecture. Contribution/Results: It achieves, for the first time, online novelty detection, semantic characterization, and model-level adaptive reconstruction by multiple agents within a high-fidelity dynamic military simulation environment. Experiments demonstrate a 42% improvement in planning success rate, sub-1.3-second response latency, and robust concurrent coordination among over ten agents.

Existing world models for embodied AI agents exhibit fragmentation in environmental prediction, intention recognition, and social context modeling, hindering coherent physical–social interaction. Method: We propose a unified “physical–mental” two-layer world modeling framework: a lower layer integrates multimodal perception, memory, and causal reasoning to model dynamic physical environments; an upper layer employs mental state inference to jointly represent user beliefs, goals, and social norms. The architecture enables cross-modal reasoning, long-horizon planning, and collaborative decision-making. Contribution/Results: Experiments demonstrate significant improvements in task completion rate, intention recognition accuracy, and human–robot interaction naturalness across both simulated and real-world settings. Our framework provides a scalable, human-aligned paradigm for advancing embodied intelligence toward human-like interactive capabilities.

Real-world deployment of intelligent agents is often hindered by scarce training data and the difficulty of constructing high-fidelity simulation environments. To address this, we propose IMAC, the first framework to integrate Unsupervised Environment Design (UED) into world-model-based imagined environments for adaptive, curriculum-driven training from offline data. IMAC synergistically combines world models, UED, and reinforcement learning to conduct progressive, curriculum-style imagination training within procedurally generated latent spaces. Experiments demonstrate that even a lightweight world model trained solely on narrow-domain offline data enables strong zero-shot transfer to unseen environments—validating the feasibility of leveraging compact world models for generalizable agent training. Our core contribution lies in pioneering the joint use of UED and world models for automated curriculum generation, significantly enhancing cross-environment generalization performance.

High-quality multi-step GUI interaction trajectories for training GUI agents are scarce and prohibitively expensive to annotate manually. Method: This paper proposes a web-tutorial-based automated trajectory synthesis framework: (1) crawling open-source online tutorials and parsing them into structured, multi-step task specifications; (2) orchestrating a vision-language model (VLM) agent to execute tasks and generate trajectories in real GUI environments; and (3) employing a VLM-based evaluator for end-to-end automatic trajectory validation. We introduce “guided replay”—the first paradigm enabling fully automated conversion of unstructured textual tutorials into executable, verifiable GUI trajectories without human annotation. Contribution/Results: Experiments demonstrate that synthesized trajectories significantly improve agent performance in GUI element localization and multi-step planning, outperforming prior methods across multiple benchmarks. Moreover, the per-trajectory data cost is reduced by over an order of magnitude, enabling scalable, low-cost GUI agent training.

This study addresses the inefficiency and inaccuracy in human error identification during the supervision of intelligent agent behavior trajectories, often caused by information overload or insufficiency. Through three user studies, the authors systematically evaluate the verification utility of baseline trajectories, explore three alternative design approaches, and propose a novel interactive interface. The interface significantly reduces the time users require to locate errors and enhances their decision confidence, though it does not yield a statistically significant improvement in final judgment accuracy. The findings uncover critical challenges in human–AI collaborative verification, including users’ implicit assumptions about agent behavior and their subjective criteria for correctness, thereby offering empirical insights to inform future designs of explainable AI systems and human–agent collaboration.

This work proposes Agent-Authored World Modeling (AAWM), a novel approach that addresses the misalignment between conventional world models—typically trained to predict future observations—and the dynamic representations required for effective agent decision-making. Instead of passively forecasting observations, AAWM enables the agent to actively retrieve relevant transitions from its experience trajectories and synthesize decision-oriented dynamics tailored to its policy needs. By leveraging a large language model to generate supervision signals aligned with strategic objectives, AAWM reframes world model training as an active, policy-driven process. Empirical results across diverse environments and training settings demonstrate that AAWM significantly outperforms traditional methods, confirming that its learning targets more effectively support policy optimization.

This work addresses the challenge of enabling agents to implicitly convey internal state information through their actions in communication-constrained environments, thereby facilitating accurate external observation. The authors propose a method that directly embeds state observability into the reinforcement learning reward function, guiding the policy to actively expose informative state signals while preserving primary task performance. By integrating reinforcement learning with observability-aware optimization, the approach successfully trains control policies with high observability in an aerial tracking task. Experimental results demonstrate that the resulting policies significantly enhance the accuracy of state reconstruction by external observers, with negligible degradation to the main task performance.

Existing single-prompt agents in 3D modeling lack planning and reflective capabilities, struggling to simultaneously achieve high geometric accuracy, aesthetic quality, and task completion rates. This work proposes a Planner-Actor-Critic multi-agent framework that introduces, for the first time, a structured self-reflection mechanism. By integrating real-time human guidance with the Blender MCP toolchain, the framework establishes a collaborative closed loop among planning, execution, and critique. This approach enables synchronous human-AI co-modeling and significantly outperforms baseline methods across diverse scenarios, demonstrating marked improvements in geometric precision, aesthetic quality, and task success rate, along with a substantial reduction in error frequency.

This work addresses the limited generalization of embodied agents in novel environments and the inefficiency of existing environment generation methods, which lack feedback mechanisms tied to agent performance. To overcome these limitations, the authors propose a closed-loop, adaptive environment generation framework that translates fine-grained performance evaluations into environmental modification signals, dynamically adjusting both the difficulty and type of challenges in training scenarios. By integrating controllable environment representations with procedural generation techniques, the framework synthesizes training environments tailored to the agent’s current learning needs. Experimental results demonstrate that this approach significantly improves training efficiency and enhances generalization performance in previously unseen environments.