From Scene Graphs to Answers: Selective Neuro-Symbolic Reasoning for Autonomous Driving

2026-09-26

AI Summary

This study addresses the lack of symbolic precision and semantic interpretability inherent in purely neural reasoning for autonomous driving question answering. To this end, it proposes a hierarchical spatiotemporal scene graph coupled with a selective neuro-symbolic framework. Through a query-adaptive mechanism, the method prioritizes symbolic reasoning and invokes large language models (LLMs) solely when necessary for targeted semantic parsing, thereby transforming LLMs from general-purpose engines into precise reasoners. Experimental results demonstrate that the proposed framework achieves an accuracy of 80.63% on the NuScenes-QA dataset, outperforming pure LLM baselines by 5.48 percentage points and yielding substantial improvements on counting tasks.

Abstract

Autonomous-driving question answering requires reasoning over structured scene information, yet existing vision-language approaches largely delegate heterogeneous reasoning operations to a single neural inference process. We argue that this uniform strategy overlooks a fundamental distinction: some queries admit exact symbolic solutions, while others require semantic interpretation. We introduce a query-adaptive neuro-symbolic reasoning framework that explicitly allocates computation according to the nature of the query. At its core is a hierarchical Spatiotemporal Scene Graph (STSG) that separates persistent object identities from frame-specific states and represents spatial relations and temporal transitions as explicit directed structures. Given a query, a symbolic executor first attempts to resolve it through exact graph operations; only when symbolic execution abstains is an LLM invoked for semantic reasoning. For these unresolved queries, query-conditioned graph retrieval and evidence filtering preserve relation direction, temporal locality, and object semantics, providing the LLM with compact and verified task-relevant evidence. This design shifts the role of the LLM from a universal reasoning engine to a targeted semantic reasoner, while allowing deterministic computation to be handled exactly and efficiently. We evaluate the framework on 5,916 NuScenes-QA questions across all ten scenes of nuScenes v1.0-mini under an oracle-perception setting. The complete system achieves 80.63 percent overall accuracy with GPT-5.4-mini, improving over the corresponding LLM-only configuration by 5.48 percentage points; with DeepSeek-V4-Flash, the improvement reaches 6.64 points. The largest gains occur on counting questions, with improvements of 10.20 and 12.61 points, respectively. These results show that selective reasoning improves both accuracy and inference efficiency.

Problem

• Autonomous Driving Question Answering
• Neuro-Symbolic Reasoning
• Scene Graphs
• Vision-Language Models

Innovation

• Neuro-Symbolic Reasoning
• Spatiotemporal Scene Graph
• Autonomous Driving QA
• Selective Reasoning
• Query-Conditioned Graph Retrieval