action space engineering

This paper addresses foundational challenges in advancing large language models toward autonomous agents—specifically, the formal definition of autonomy, decision-making mechanisms, and goal hierarchies. Method: It introduces the principle “knowledge boundary equals tool-use boundary,” unifying internal reasoning and external tool invocation as equivalent cognitive operations; cognitive action boundaries are rigorously defined by knowledge accessibility, enabling optimal cognitive efficiency. Integrating cognitive modeling, knowledge representation, and decision theory, the approach constructs a unified, interpretable cognitive framework tailored for foundation agents. Contribution/Results: The framework enables adaptive, goal-directed, and minimally redundant tool invocation. It establishes the first theoretically grounded, architecture-agnostic cognitive paradigm for autonomous agents—providing both rigorous formal foundations and a scalable design blueprint for next-generation intelligent systems.

This paper addresses the weak adaptability, poor generalization, and insufficient long-term stability of multimodal large language model (MLLM)-driven GUI agents in complex real-world environments. We propose a reinforcement learning (RL)-based enhancement framework that formalizes GUI interaction as a Markov decision process and systematically decomposes the agent’s evolution across perception–planning–execution modules. We introduce, for the first time, a three-tiered taxonomy of GUI agent training paradigms—prompt engineering, supervised fine-tuning, and RL—highlighting the critical transition from static response generation to dynamic policy learning. Leveraging GUI environment simulation and modular architecture design, we empirically validate RL’s central role in improving cross-application generalization, robustness, and long-horizon task stability. The work establishes the first comprehensive RL-enhanced theoretical framework and practical guidelines tailored specifically for GUI agents.

Autonomous AI systems struggle to simultaneously satisfy procedural, legal, and ethical constraints in real-world environments, leading to decision-making dilemmas under conflicting norms. Method: We propose a dynamic decision-making framework that integrates normative, pragmatic, and contextual knowledge. It enables AI to autonomously generate candidate actions under constraint conflicts, evaluate them across multiple dimensions—including goal consistency and value alignment—and produce human-interpretable action justifications. Technically, the framework unifies multi-source knowledge reasoning, fine-grained situational understanding, and explainable planning—moving beyond limitations of end-to-end policy learning. Contribution/Results: This work is the first to systematically characterize the knowledge types and mechanisms required for compliant and reasonable decision-making in underspecified scenarios. Empirical evaluation demonstrates substantial improvements in behavioral planning robustness, contextual adaptability, and alignment with human values.

In high-stakes domains such as clinical decision-making, AI model explanations often lack verifiability, undermining practitioner trust. To address this, we propose the Action-based Reasoning Agent (ARA), which formalizes diagnostic explanations as auditable action sequences. ARA actively retrieves external visual evidence via reinforcement learning and incorporates a causal intervention mechanism to ensure explanation faithfulness and traceability. Our key contributions are: (1) explicit decomposition of the explanation process into verifiable, executable actions; and (2) quantification of the causal contribution of explanations to model decisions via evidence-masking experiments. Evaluated on medical image diagnosis tasks, ARA significantly improves calibration accuracy over non-interactive baselines, reducing the Brier score by 18%. Critically, masking pivotal evidence increases the Brier score by 0.029—empirically validating both the faithfulness and necessity of the generated explanations.

Current AGI development is constrained by a theoretical bottleneck—observational learning cannot guarantee intervention competence, as observationally equivalent worlds may exhibit divergent behaviors under interventions. Method: We propose a novel paradigm of intelligence centered on “error,” reconstructing learning and intelligence through critical rationalism and establishing a causal mechanics framework. This framework introduces differential locality, gauge-invariant independent causal mechanisms, and compositional autonomy—enabling dynamic expansion of hypothesis spaces and rendering otherwise unreachable errors accessible. Contribution: We provide the first systematic characterization of the fundamental limitations of observational learning and establish a foundational theory of intelligence that transcends observational equivalence. Furthermore, we deliver operational tools for error diagnosis, counterfactual reasoning, and synthesis of causal mechanisms—laying a formal foundation and design pathway for self-correcting, self-evolving AGI systems.

This work addresses the high barrier to entry in existing imitation learning methods, which typically require extensive expert demonstrations and continuous monitoring, rendering them inaccessible to non-expert users. To overcome this limitation, we propose an interactive policy restructuring and training framework that enables ordinary users without machine learning expertise to dynamically adjust an agent’s policy structure and parameters through natural-language instructions and a minimal number of demonstrations. Our approach integrates instruction parsing, dynamic policy architecture evolution, online imitation learning, and human-in-the-loop interaction mechanisms. A user study (N=34) demonstrates that, compared to baseline methods, the proposed framework significantly enhances policy robustness and trustworthiness while maintaining high usability.

Existing large language model agent systems struggle to meet the demands of production environments—such as simplicity, controllability, and predictable inference costs—due to their high complexity, unbounded reasoning expenses, and unpredictable behavior. To address these limitations, this work proposes a practical, utility-driven agent design framework that employs “pseudo-tools” to enforce modularity, replaces dynamic planning with fixed workflows, and integrates a dedicated learning algorithm to jointly optimize component performance. The approach innovatively applies multi-objective optimization to balance inference cost and response quality, while supporting result fusion across multiple systems. Experimental results demonstrate that the proposed method significantly reduces inference costs and improves accuracy across diverse tasks, outperforming handcrafted dynamic planning baselines.

This work addresses key challenges faced by GUI agents in long-horizon tasks—namely credit assignment, distributional shift, and safe exploration—by systematically integrating reinforcement learning (RL) methodologies into a unified framework encompassing offline, online, and hybrid strategies. Through the incorporation of multi-level reward mechanisms, world models, and cognitive architectures, the study outlines a developmental pathway for digitally native agents. It presents the first comprehensive survey on the integration of RL with GUI agents, clarifying core directions for enhancing agent reliability and scalability. The proposed framework provides theoretical foundations for building robust automation systems and agent-native infrastructure, advancing the frontier of intelligent interaction in graphical user environments.

Traditional agent models are constrained by synchronous action–observation interfaces operating at fixed time steps, making it difficult to capture the asynchronous, multi-scale temporal interactions prevalent in real-world environments. This work proposes the Engagement Process (EP) framework, which decouples actions and observations into independent temporal event streams while preserving the theoretical foundations of POMDPs, thereby explicitly incorporating time as a first-class dimension. The framework naturally supports modeling complex behaviors such as delayed feedback, continuous actions, multi-rate coordination, and compositional interactions among subsystems. Empirical evaluations demonstrate that EP effectively uncovers temporal dynamics obscured by conventional step-based interfaces across toy tasks, large language model agents, and learning scenarios, enabling efficient policy adaptation under explicit time-cost constraints.

This work addresses the limitations of existing evaluation methods, which focus narrowly on task completion and fail to ensure trustworthy deployment of embodied agents in multi-step, externally impactful scenarios, while also lacking coordination among evaluation, governance, orchestration, and runtime assurance. To bridge this gap, the paper proposes an integrated four-layer framework that establishes, for the first time, a closed-loop mechanism linking governance obligations to verifiable execution. Guided by the ODTA principles—Observability, Decidability, Timeliness, and Attestability—the framework introduces runtime localization testing and minimal action evidence bundles. Through a human-in-the-loop evidence synthesis approach, it formally connects policy requirements to concrete agent behaviors, exposing critical gaps such as the inability of static permissions and prompts to govern path-dependent actions. Validation via an enterprise procurement agent demonstrates the framework’s capacity to unify safety, robustness, and trajectory-level evaluation.