state representation design

Designing how system state is represented, checkpointed, versioned, and propagated so downstream agents can act on information they cannot re-observe. Used to formulate MDP states for control or DRL, manage distributed state in managers/DAGs, and ensure reliable information flow under stochastic dynamics.

staterepresentationdesign

12-Month Skill Trend

Momentum and market value over time
Trending
Score
+20 in 12 mo
96
12 mo agoNow
Career
Value
+$12K in 12 mo
$42K/year
12 mo agoNow

Recommended Survey Paper

Quick overview of the field
View more

Must-Read Papers

Most classic and influential ideas
View more

CRDT Emulation, Simulation, and Representation Independence

Apr 07, 2025
NL
Nathan Liittschwager
🏛️ University of California, Santa Cruz | University of Southern Denmark

This paper addresses the paradigmatic equivalence between state-based and operation-based CRDTs—a longstanding challenge arising from their differing dependencies on network causality and message granularity, with prior work lacking a rigorous formalization of their simulation relationship. To resolve this, we introduce a novel weak simulation framework based on labeled transition systems extended with network interactions, precisely defining cross-paradigm simulation semantics and establishing necessary conditions for property preservation. We prove that, under causal consistency, no client can distinguish between state-based and operation-based implementations—thereby achieving representation independence. Our results formally delineate the equivalence boundary between the two paradigms, enabling correctness-preserving cross-paradigm property transfer, verification reuse, and flexible system design. This work provides a unified semantic foundation for both CRDT theory and practical implementation.

Characterizing preserved properties in CRDT weak simulationsEstablishing representation independence for CRDT client interactionsFormalizing emulation between state-based and op-based CRDTs

Interpreting systems as solving POMDPs: a step towards a formal understanding of agency

Sep 04, 2022
MB
Martin Biehl
🏛️ Cross Labs | Cross Compass | Earth-Life Science Institute | Tokyo Institute of Technology

This paper addresses the foundational question: *When can a physical system be rigorously regarded as an agent possessing beliefs and goals?* We propose a POMDP-based explanatory framework, formalizing an agent as a physical system satisfying two joint constraints: (i) its state evolution must conform to Bayesian belief-updating dynamics, and (ii) its policy must be optimal with respect to a specified objective. Crucially, we introduce the *completeness of a POMDP solution*—simultaneous adherence to correct belief evolution and optimal action selection—as a necessary and empirically falsifiable criterion for agency, overcoming the limitations of prior definitions relying solely on belief mapping. This yields the first axiomatization of agency that is mathematically rigorous, computationally operational, and empirically testable. It reveals necessary constraints linking physical dynamics to agential properties, thereby establishing a theoretical foundation for AI safety verification and computational modeling of consciousness.

Can systems be interpreted as solutions to POMDPs?How do agency features relate to physical system states?When can a system be considered to have beliefs and goals?

Traditional distributed systems struggle to support modern autonomous infrastructures that integrate stochastic models and autonomous agents. This work proposes the Post-Deterministic Distributed System (PDDS) model, introducing for the first time its five architectural pillars. Its core innovation is a "cognitive state replication" mechanism that extends consistency from data visibility to knowledge visibility, alongside a novel fault classification framework. By leveraging protocol-driven development, verifiable agent infrastructure, and semantic quorum guarantees, PDDS enables coordination among semantically equivalent yet executionally divergent agents. This approach achieves verifiable semantic rollback and cross-agent reasoning consistency, establishing a theoretical foundation for trustworthy autonomous systems.

Autonomous AgentsDistributed ConsistencyPost-Deterministic Distributed Systems

Good-for-MDP State Reduction for Stochastic LTL Planning

Nov 12, 2025
CW
Christoph Weinhuber
🏛️ University of Oxford | Key Laboratory of System Software (Chinese Academy of Sciences) | Institute of Software Chinese Academy of Sciences | University of Liverpool

To address the scalability bottleneck in MDP policy synthesis under LTL objectives—caused by state-space explosion during generalized Rabin automaton (GFM) construction—this paper proposes a novel state-space reduction framework. Methodologically, it introduces (1) a game-theoretically optimal “good-for-games minimization” technique, integrating formal translation with specialized GFM construction, and (2) for the key LTL fragment $mathsf{G}mathsf{F}varphi$, a direct GFM construction algorithm achieving single-exponential time complexity, breaking the classical double-exponential barrier. Experimental evaluation on standard benchmarks demonstrates that the approach reduces automaton size by one to two orders of magnitude, significantly improving policy synthesis efficiency and overall scalability. These advances provide a practical pathway for large-scale LTL-constrained MDP planning.

Creating specialized construction for GFφ formulas with better complexityDeveloping efficient state-space reduction for good-for-MDP automataReducing automata state size for scalable LTL planning in MDPs

This work addresses the scalability bottleneck in Markov decision processes (MDPs) caused by the exponential explosion of state spaces by proposing a property-oriented causal abstraction framework. It introduces causal reasoning into MDP abstraction for the first time, leveraging causal relationships among predicates over state variables to cluster states that either satisfy or violate a given target property, thereby constructing compact yet property-preserving abstract models. The framework accommodates various model types—including MDPs, interval MDPs, and stochastic games—and generates small abstract models on standard benchmarks that enable efficient computation of near-optimal policies. Empirical evaluation demonstrates strong generalization across large-scale MDPs, effectively balancing model fidelity with compression efficiency.

abstractioncausalityMarkov Decision Processes

Latest Papers

What's happening recently
View more

Existing autonomous systems lack enforceable guarantees on the permissibility of state transitions in concurrent environments, often leading to uncontrolled or non-compliant changes. This work introduces the concept of an “atomic decision boundary,” which tightly couples policy evaluation with state transition into an indivisible operation. Formalized using labeled transition systems (LTS), the approach distinguishes atomic systems from those employing decoupled evaluation. The study demonstrates, for the first time, that under concurrent semantics, decoupled evaluation is not equivalent to atomic execution and that classical TOCTOU (Time-of-Check-to-Time-of-Use) analyses overlook a critical “Escalate” scenario requiring atomicity. It further proves that the atomic decision boundary constitutes a necessary structural condition for ensuring permissibility across all execution traces, thereby establishing a theoretical foundation for runtime governance. This paper is the inaugural contribution to the Agent Governance series.

admission controlatomic decision boundaryconcurrent systems

Existing adaptive agent-based regulatory simulations lack mechanisms to systematically integrate diagnostic feedback into policy controllers, resulting in delayed and opaque policy adjustments. This work proposes a lightweight machine-guided policy revision layer that represents policies as defeasible rules and combines symbolic control, defeasible logic, and policy prioritization to operationalize contestability at the controller level, thereby endowing policy decisions with explainability, contestability, and dynamic revisability. In an emissions regulation agent-based model, the approach significantly reduces problem recurrence under scenarios where the VPVA mechanism fails due to excessive conservatism, while effectively maintaining key performance indicators such as violation rates, overshoot, and volatility.

adaptive simulationagent-based modelingcontestability

This work proposes an evidence-centered denotational semantic framework that reframes distributed consensus by modeling execution as higher-order evidence states induced by communication, with consensus outputs emerging as their first-order projections. Traditional output-centric descriptions of consensus protocols obscure the underlying evidential structures formed through communication; this framework unifies protocol differences as design choices concerning valid evidence, observational horizons, collapse timing, and repair mechanisms. By formalizing “collapse strategies” and integrating insights from the FLP impossibility result and topological distributed computing theory, the approach reveals a fundamental connection between consensus protocols and their evidential underpinnings, offering a novel paradigm for the analysis, comparison, and design of such protocols.

collapse policycommunication evidenceconsensus

This work addresses the problem of policy synthesis in Markov decision processes (MDPs) under entropy-based constraints that enforce concentration of state visitation distributions. It formalizes entropy maximization as a policy synthesis objective for the first time, establishes its computational complexity, and introduces a novel method combining convex duality theory with invariant synthesis to handle nonlinear entropy constraints in a conditionally complete manner. By systematically analyzing the roles of memory and randomization in policies, the approach effectively synthesizes and verifies entropy-constrained policies across multiple benchmark instances, substantially extending the expressiveness and applicability of existing policy synthesis frameworks.

concentration propertycontrol policy synthesisentropy objectives

This work addresses the challenge of coupled signaling and control in partially observable Markov decision processes (POMDPs) by proposing an information-theoretic meta-dynamic programming framework under average-cost constraints. By introducing a dual-coupled information state—comprising the posterior distribution of the system state and the distribution over this posterior as sufficient statistics—the optimal stochastic control policy is decomposed into a separated structure dependent solely on these two information states, and necessary and sufficient conditions for optimality are established. The approach integrates directed information measures, Bayesian recursions, and dynamic programming over the probability simplex, naturally reducing to the classical POMDP solution in the absence of communication. This framework provides the first analytically tractable optimal control solution for POMDPs with endogenous information constraints, laying a theoretical foundation for integrated communication-control intelligent systems.

controldirected informationinformation-theoretic

Hot Scholars

NY

Nobuko Yoshida

Department of Computer Science, University of Oxford
Programming Languages and SystemsTheoretical Computer ScienceConcurrency TheoryVerification
BF

Bernd Finkbeiner

Professor of Computer Science, CISPA Helmholtz Center for Information Security
Reactive SystemsVerificationSynthesisTemporal Logic
SJ

Sebastian Junges

Assistant Professor, Radboud University, Nijmegen
Formal methodsMarkov Decision ProcessesController SynthesisProbabilistic Inference
MZ

Martin Zimmermann

Aalborg University
computer scienceverificationautomata theorytemporal logics
XQ

Xue Qin

Assistant Professor, Villanova University
Software EngineeringSoftware Testing