Problem
Research questions and friction points this paper is trying to address.
edge devices
small language models
reliable reasoning
resource-constrained
neurosymbolic
Neurosymbolic Routing for Reliable Reasoning on Resource-Constrained Edge Devices
🤖 AI Summary
This study addresses the low reliability and high energy consumption of small language models (SLMs) on edge devices when handling structured tasks. We propose a neuro-symbolic router that leverages the L* algorithm, employing the SLM as a membership oracle, to automatically learn deterministic finite automata (DFA) as routing logic. Without manual encoding, this approach precisely dispatches queries: structured tasks are routed to a deterministic engine, while open-ended questions are handled by the SLM. Evaluated on a Raspberry Pi, the proposed framework achieves 100% routing accuracy and 98.3% overall precision, yielding an 8.8× improvement in inference speed and a 2.8× gain in energy efficiency.
📝 Abstract
Running a language model on edge hardware provides private and low-latency reasoning without a network connection, and yet the small models that fit on such devices are unreliable on the tasks computers are expected to handle well, such as arithmetic, algebra, and formal logic problems. We argue that much of this unreliability is avoidable. Many queries appearing to demand reasoning are in fact structurally deterministic and permit fast and exact symbolic solutions. Therefore, forcing a probabilistic model to approximate them sacrifices accuracy and energy for little benefit. We present a neurosymbolic router that classifies each incoming query and dispatches it to the cheapest correct solver, sending structured tasks to deterministic engines and reserving the small language model (SLM) for open-ended word problems. Instead of hand-coding the routing logic, we learn a deterministic finite automaton (DFA) with the L* grammatical inference algorithm, using the SLM as a membership oracle and labeled data as an equivalence oracle. On a Raspberry Pi 4B (8 GB RAM, no GPU), evaluated on 100 untested prompts from DeepMind Mathematics, GSM8K, and RuleTaker, learned routing attains 100% routing accuracy and 98.3% overall accuracy with a 512-token reasoning budget (93.3% on word problems), compared with 72.0% for the strongest agent baseline, Program-of-Thought, and 58.7% for a tool-calling agent given the same solvers. Since formatted queries never reach the model, the router answers them in 1-11 ms and, in its 30-token configuration, runs 8.8x faster and 2.8x more energy-efficient than Program-of-Thought.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Neurosymbolic Routing
Deterministic Finite Automaton
L* Algorithm
Small Language Model
Edge Devices
🔎 Similar Papers
No similar papers found.
