Technology Category
Knowledge Representation and Reasoning: Qualitative ReasoningMachine Learning: Reinforcement LearningReasoning under Uncertainty: Sequential Decision Making
Interpretable by Design: Query-Specific Neural Modules for Explainable Reinforcement Learning
🤖 AI Summary
Traditional reinforcement learning (RL) implicitly encodes environmental knowledge—such as reachability, distances, and dynamics—making it difficult to explicitly query or interpret such information. Method: We reformulate RL agents as general-purpose reasoning engines capable of answering diverse environmental queries—including policy execution, reachability, path planning, and comparative analysis—rather than solely optimizing control policies. To this end, we propose Query-conditioned Deterministic Inference Networks (QDIN), a neural architecture that decouples reasoning from control via query conditioning and modular design, enabling high-fidelity environmental inference within a unified deterministic framework. Contribution/Results: Experiments demonstrate QDIN achieves 99% IoU accuracy on reachability tasks and attains 31% normalized return on control benchmarks—substantially outperforming monolithic models and post-hoc extraction methods. This work provides the first empirical validation that high-fidelity environmental reasoning can be learned independently of optimal policy learning, advancing a queryable, interpretable, knowledge-base–driven RL paradigm.
Technology Category
Application Category
📝 Abstract
Reinforcement learning has traditionally focused on a singular objective: learning policies that select actions to maximize reward. We challenge this paradigm by asking: what if we explicitly architected RL systems as inference engines that can answer diverse queries about their environment? In deterministic settings, trained agents implicitly encode rich knowledge about reachability, distances, values, and dynamics - yet current architectures are not designed to expose this information efficiently. We introduce Query Conditioned Deterministic Inference Networks (QDIN), a unified architecture that treats different types of queries (policy, reachability, paths, comparisons) as first-class citizens, with specialized neural modules optimized for each inference pattern. Our key empirical finding reveals a fundamental decoupling: inference accuracy can reach near-perfect levels (99% reachability IoU) even when control performance remains suboptimal (31% return), suggesting that the representations needed for accurate world knowledge differ from those required for optimal control. Experiments demonstrate that query specialized architectures outperform both unified models and post-hoc extraction methods, while maintaining competitive control performance. This work establishes a research agenda for RL systems designed from inception as queryable knowledge bases, with implications for interpretability, verification, and human-AI collaboration.
Problem
Research questions and friction points this paper is trying to address.
Designing RL systems as queryable inference engines for environmental knowledge
Developing specialized neural modules for different query types like reachability and paths
Decoupling inference accuracy from control performance in reinforcement learning
Innovation
Methods, ideas, or system contributions that make the work stand out.
Query-specific neural modules for explainable reinforcement learning
QDIN architecture treats diverse queries as first-class citizens
Specialized modules optimized for different inference patterns
🔎 Similar Papers
No similar papers found.
