state representation

This work addresses the lack of systematic, data-driven analysis of reinforcement learning (RL) environment evolution, which has obscured the developmental trajectory from physics-based simulation toward general-purpose agents. To bridge this gap, we propose the first data-driven, multidimensional taxonomy framework that integrates automated semantic analysis, statistical modeling, and large-scale literature mining to systematically examine over 2,000 core papers. Our analysis reveals an ongoing paradigm shift in RL environments, bifurcating into two distinct ecosystems—“semantic priors” and “domain generalization”—and for the first time characterizes their cognitive attributes. We identify key underlying mechanisms, including cross-task synergy, multi-domain interference, and zero-shot generalization, thereby offering a foundational design blueprint for next-generation embodied semantic simulators.

Traditional general intelligence research overemphasizes external reward maximization, resulting in agents with limited adaptability and poor generalization. This paper proposes an agent-centric learning paradigm that shifts the learning objective from environmental control to the controllable construction and dynamic diversification of internal knowledge representations. Its core contribution is the formalization—first in the literature—of “representation empowerment”: the agent’s capacity to actively shape its own knowledge representations, thereby transcending dependence on extrinsic feedback. Methodologically, the framework integrates intrinsic-motivation-driven reinforcement learning with representation learning, introducing a joint metric of controllability and diversity to support self-organized knowledge development. Experiments demonstrate substantial improvements in cross-task generalization and environmental adaptability. The approach provides a novel, interpretable, and controllable pathway for designing general intelligence systems.

This work addresses the challenge of adapting traditional reinforcement learning to out-of-distribution, complex, or long-horizon tasks, where reliance on low-level action spaces hinders effective generalization. To overcome this limitation, the paper introduces Semantic Action Reinforcement Learning (SARL), which leverages learnable language prompts as a semantic action space. SARL enables online optimization of these prompts to modulate pretrained skill compositions within a vision–language–action (VLA) generalist policy. This approach facilitates structured exploration and efficient online adaptation, significantly outperforming existing methods. Empirical results demonstrate that SARL successfully unlocks the capability of general-purpose robotic policies to handle novel tasks in both simulated and real-world environments.

This work addresses the challenge of effectively integrating multi-source, multi-stage semantic information to establish a stable perception-action loop for embodied intelligence in complex, open environments. To overcome the limitations of traditional modular paradigms, the paper proposes a unified “Semantic Lifecycle” framework that conceptualizes semantic knowledge acquisition, representation, and storage as a continuous and dynamic knowledge evolution process. Leveraging the cross-domain generalization capabilities and semantic priors of foundation models, the framework integrates multimodal perception, knowledge representation, and memory mechanisms. The study systematically reviews and contrasts recent advances across the three stages of the semantic lifecycle, identifies key challenges, and provides a cohesive theoretical foundation to advance embodied intelligence toward greater generality and robustness.

This study addresses the fragmentation between vectorized learning and knowledge graph representation in cognitive agents, as well as overreliance on large language models and ontology-based modeling. Methodologically, it introduces an ontology-free attention mechanism grounded in commitment theory semantics: (i) a semantic spatiotemporal γ(3,4) graph serves as a unified representational framework, with commitment theory explicitly embedded into attention computation to enable role-driven knowledge categorization; (ii) a causally bounded attention compression paradigm reduces contextual data requirements. Contributions include: (i) the first formal integration of commitment theory into attention semantics, enabling causal reasoning under uncertainty; (ii) order-of-magnitude context compression in autonomous robotics, defense, and emergency response applications—while preserving statistical stability, source-intent fidelity, and real-time inference robustness.

This work addresses the limited generalization of existing object search methods, which rely heavily on large amounts of in-domain training data and struggle in unseen environments. To overcome this challenge, the authors propose an autonomous search framework that integrates semantic priors derived from human experts. Specifically, they model semantic priorities by simulating expert guidance and embed this model into a combinatorial optimization–based frontier exploration planner. This approach is the first to incorporate high-level semantic relationships into object search, significantly enhancing both generalization capability and search efficiency in novel environments. Simulation experiments demonstrate that, compared to purely coverage-driven baseline methods, the proposed framework achieves full environmental coverage while substantially accelerating target discovery.

This study addresses a fundamental limitation in mainstream artificial intelligence, which typically treats cognition as internal computation decoupled from the body and environment, thereby overlooking the inseparability of perception and action, embodiment, autonomy, and the dynamic, interactive nature of experience. For the first time, this work systematically introduces the four core principles of enactive cognition—sensorimotor coupling, experience-dependence, embodiment, and intrinsic normativity—into AI research, aligning them with reinforcement learning frameworks to reveal both structural resonances and theoretical gaps. The paper not only evaluates how closely existing AI systems approximate these enactive principles but also proposes concrete pathways for their integration, laying a theoretical foundation for developing next-generation agents that are embodied, interactive, and genuinely agentic, thereby advancing AI toward forms of intelligence more closely aligned with biological cognition.

Existing methods struggle to reliably attribute goals in agent systems, limiting the interpretability and predictability of their behavior. This work proposes an integrated framework that combines behavioral evaluation with internal representation analysis to investigate the goal-directedness of language model agents navigating toward target states in 2D grid environments. Through behavioral benchmarks, comparisons with optimal policies, representation probing, and reasoning process analysis, we find that agent performance scales robustly with task difficulty, that agents coarsely encode the spatial structure of the environment, and that their reasoning processes dynamically shift representations from reliance on global cues toward supporting immediate actions. These findings reveal how language model agents nonlinearly encode spatial information and adaptively refine their internal representations to enable goal-directed decision-making.

To bridge the semantic gap between natural language instructions and autonomous robotic execution in unstructured environments, this paper proposes the first end-to-end vision-language-policy joint model that unifies multimodal perception, semantic understanding, and dynamic behavior planning. Methodologically, a vision-language model (VLM), fine-tuned on real-world data, is tightly coupled with modules for scene understanding, instruction parsing, and policy generation; it is trained via a synergistic combination of reinforcement learning and imitation learning. The resulting framework enables real-time replanning upon in-execution instruction changes and exhibits cross-robot embodiment generalization. Experiments demonstrate a policy update latency under 0.8 seconds, cross-platform transfer success rate exceeding 89%, and significantly improved adaptability to novel tasks and responsiveness to dynamic environmental changes.

This work addresses the challenge of efficiently transmitting task-relevant semantic information under bandwidth constraints in multi-agent collaborative perception, where decisions must be both explainable and verifiable for safety-critical applications. The authors propose an interpretable semantic communication framework grounded in a hierarchical world representation based on first-order logic (FOL). For the first time, they integrate FOL structures with semantic information measures, defining semantic entropy and mutual information under logical probability, and introduce a goal-oriented state layer to abstract decision-critical information. By leveraging semantic rate-distortion theory and the semantic information bottleneck principle, the method selects the most informative FOL clauses for transmission, achieving semantic compression while preserving logical verifiability. Evaluated in a dynamic urban multi-agent simulation environment, the approach significantly reduces communication overhead while maintaining high decision accuracy and logical interpretability.

This work addresses the inherent tension among autonomy, robustness, and governability in large language model (LLM)-driven agents. To reconcile these competing demands, the authors propose the “Cartesian Agent” architecture, which explicitly decouples control states from policy through a symbolic interface, thereby coupling the predictive capabilities of LLMs with an engineered runtime system. The framework incorporates hierarchical feedback loops and runtime intervention mechanisms, and formally distinguishes three control paradigms: bounded services, Cartesian agents, and integrated agents, highlighting how the locus of control critically influences system performance. Experimental results demonstrate that the Cartesian architecture offers significant advantages in modularity, governability, and startup efficiency, while also revealing limitations related to sensitivity and potential bottlenecks.