🤖 AI Summary
This work addresses the trade-off between reliability and cost in large language model (LLM) agents for tool invocation: fully LLM-driven routing incurs high computational overhead, while static workflows are prone to silent failures under compound tool errors. To overcome this, we propose the Self-Healing Router architecture, which introduces runtime fault tolerance into graph-based tool-calling systems for the first time. By modeling control flow as a weighted graph routing problem, our approach employs parallel health monitoring to dynamically assess tool status and leverages Dijkstra’s algorithm for deterministic shortest-path rerouting, invoking the LLM only when necessary. Evaluated across 19 scenarios, the method achieves accuracy comparable to ReAct while reducing control-plane LLM calls by 93% (from 123 to 9) and completely eliminating silent failures, thereby enabling binary observability and efficient autonomous recovery.
📝 Abstract
Tool-using LLM agents face a reliability-cost tradeoff: routing every decision through the LLM improves correctness but incurs high latency and inference cost, while pre-coded workflow graphs reduce cost but become brittle under unanticipated compound tool failures. We present Self-Healing Router, a fault-tolerant orchestration architecture that treats most agent control-flow decisions as routing rather than reasoning. The system combines (i) parallel health monitors that assign priority scores to runtime conditions such as tool outages and risk signals, and (ii) a cost-weighted tool graph where Dijkstra's algorithm performs deterministic shortest-path routing. When a tool fails mid-execution, its edges are reweighted to infinity and the path is recomputed -- yielding automatic recovery without invoking the LLM. The LLM is reserved exclusively for cases where no feasible path exists, enabling goal demotion or escalation. Prior graph-based tool-use systems (ControlLLM, ToolNet, NaviAgent) focus on tool selection and planning; our contribution is runtime fault tolerance with deterministic recovery and binary observability -- every failure is either a logged reroute or an explicit escalation, never a silent skip. Across 19 scenarios spanning three graph topologies (linear pipeline, dependency DAG, parallel fan-out), Self-Healing Router matches ReAct's correctness while reducing control-plane LLM calls by 93% (9 vs 123 aggregate) and eliminating the silent-failure cases observed in a well-engineered static workflow baseline under compound failures.