Neuro-Symbolic Meta-Policies for Temporal Knowledge-Graph Memory under Partial Observability

2026-07-20

AI Summary

This work addresses the challenge of dynamic memory management for long-horizon decision-making in partially observable reinforcement learning. The authors propose a neuro-symbolic meta-policy that operates within RoomKG, a temporal knowledge graph memory system grounded in RDF, to enable interpretable and traceable memory operations through symbolic execution of adaptive heuristic rules. The approach integrates a StarE-GNN encoder, temporal annotations on RDF triples, symbolic graph manipulations, and a multi-value head mechanism governing question answering, exploration, and forgetting. To the best of our knowledge, this is the first method to combine neuro-symbolic reasoning with temporal RDF knowledge graphs, achieving both semantic web compatibility and strong adaptability. Experimental results demonstrate that, under a memory capacity of 512, the Qualifier-aware StarE-GNN configuration significantly outperforms existing symbolic, neural, and neuro-symbolic baselines on long-horizon RoomKG tasks while supporting step-by-step traceability of decision processes.

Abstract

Partially observable reinforcement learning requires deciding what to retain, retrieve, and forget over time. We introduce a neuro-symbolic meta-policy that learns which symbolic memory heuristic to apply at each decision point while keeping execution symbolic. Our setting uses temporal knowledge-graph memory in RoomKG, where hidden state and observations are represented as Resource Description Framework (RDF) graphs and memory is augmented with temporal RDF triple annotations. The model combines knowledge-graph encoding of memory contents with value heads for question answering, exploration, and forgetting, yielding a controller that is both adaptive and inspectable. This gives the work a direct Semantic Web grounding through RDF-based representation, annotation-compatible graph semantics, and graph-based symbolic operations over explicit memory state. On train/test room splits at long-term memory capacity of 512, the qualifier-aware StarE-GNN configuration achieves the best held-out performance among the compared symbolic, neural, and neuro-symbolic systems while preserving step-level traceability of memory-management decisions.

Problem

• Partially Observable Reinforcement Learning
• Temporal Knowledge Graph
• Memory Management
• Neuro-Symbolic AI
• RDF Graphs

Innovation

• Neuro-Symbolic
• Temporal Knowledge Graph
• Partial Observability
• RDF Memory
• Meta-Policy