Topology-Aware State Abstraction with Tangle Cores for Markov Decision Processes

2026-05-29

AI Summary

Traditional state abstraction relies on hard partitions, which struggle to effectively model shared interface states—such as doors or hubs—common in navigation and hierarchical decision-making. This work introduces graph tangles into reinforcement learning for the first time, proposing the tangle-core abstraction framework: it constructs overlapping abstract states via low-order separators of the empirical transition graph and represents shared interfaces using membership kernel functions. The approach enables modeling of overlapping regions, provides value-preserving guarantees, and reveals how hard partitions induce avoidable boundary errors at interfaces. Experiments demonstrate that, across bottlenecked tabular domains, procedurally generated mazes, and MiniGrid environments, the method achieves a superior trade-off between compression and return, while also identifying failure modes when transition topology lacks informative structure.

Abstract

State abstraction in reinforcement learning is usually formulated as a partition of states based on reward and transition similarity. This excludes a common structural pattern in navigation, graph, and hierarchical decision problems: interface states such as doors, hubs, and bottlenecks naturally participate in more than one region. We introduce \emph{tangle-core abstraction}, an overlapping state-abstraction framework based on graph tangles of empirical transition graphs. The method constructs abstract states from consistently oriented low-order separations and represents shared interfaces through a membership kernel rather than a hard partition. We give value-preservation guarantees for the induced overlapping abstract MDP under an explicit action-consistency condition, identify an interior-homogeneity/boundary-leakage error decomposition, and prove a quantitative interface-overlap result showing when hard partitions incur an avoidable boundary error. Empirically, tangle-core abstractions achieve favorable compression--return tradeoffs against reward-aware, learned, topological-map, and graph-partitioning baselines across bottlenecked tabular domains, procedurally generated mazes, and MiniGrid representations. We also identify a clear failure regime in which transition topology is uninformative, where tangles predictably offer little benefit. These results position graph tangles as an effective topology-aware abstraction prior for decision problems with shared interface structure.

Problem

• state abstraction
• Markov Decision Processes
• graph tangles
• overlapping partitions
• interface states

Innovation

• tangle-core abstraction
• overlapping state abstraction
• graph tangles
• topology-aware MDP
• interface states