PhyAgentOS: A Self-Evolving Operating System for Embodied Agents with Decoupled Cognitive Planning and Physical Execution

2026-07-18

AI Summary

This work addresses the lack of unified execution abstractions, shared state representations, and continual learning mechanisms in current physically embodied intelligent systems. It proposes a self-evolving operating system for embodied agents, where “conversation” serves as the fundamental unit for scheduling and verification. The system decouples cognitive planning from physical execution via a State-as-a-File protocol and integrates evidence-based semantic validation, a cognitive memory loop, and multi-layer safety constraints—including pre-checks, action bridging, and SafetyGuard. This architecture enables continual evolution without retraining and has been validated across over 19 simulated and real-world platforms, including Optimus-67, StarDojo, and DST-Dojo, demonstrating significant performance improvements for various Vision-Language-Action (VLA) models on LIBERO, Calvin, and RoboCasa365 benchmarks.

Abstract

Vision-language-action models, world models, and agentic planners each advance physical intelligence, yet their composition lacks a common execution abstraction, shared state, semantic verification, and persistent experience across heterogeneous embodiments. We present PhyAgentOS, a runtime foundation delivering scheduling, verification, memory, benchmarking, and safety as system-level services. Its Session-Centered Runtime treats a session, not an action, as the minimum unit of scheduling, compatibility preflight, supervised execution, evidence collection, and acceptance. To decouple cognition from physical execution, the cognition-physics boundary is a file system: the State-as-a-File protocol materializes cross-layer state as Markdown with YAML, yielding inspectable, versionable records without code dependencies between Agent and Runtime layers. These views form a unified cognitive state space aligning intent, capabilities, environment, execution, and experience. The SessionVerifier distinguishes execution termination from semantic task completion via evidence-grounded verdicts of success, failure, or replan. Verified outcomes are consolidated through epistemic memory into reusable knowledge and corrective lessons, closing a trial-and-error loop without retraining. Benchmarking reuses the deployment session and verification path, so results trace to real execution. Layered safety constrains both policy-driven and agent-driven execution: preflight, action bridges, SafetyGuard, heartbeat monitoring, and target-local constraints. Validation is progressive: games test cognitive planning, simulation adds dynamics and control, real robots add hardware noise, with the cognitive layer held constant. PhyAgentOS is benchmarked on Optimus-67, StarDojo, and DST-Dojo, validated on 19+ simulated and physical embodiments, and gains on LIBERO, Calvin, and RoboCasa365 across multiple VLA models.

Problem

• embodied agents
• cognitive planning
• physical execution
• execution abstraction
• persistent experience

Innovation

• PhyAgentOS
• State-as-a-File
• Session-Centered Runtime
• Epistemic Memory
• Decoupled Cognitive Planning