Think Short, Defer Smart, Act, and Repeat: Calibrated Reasoning and Uncertainty-Aware Deferral for Edge LLM Agents

2026-07-29

AI Summary

This work addresses the challenge of deploying large language model (LLM) agents at the edge, where they must execute multi-step tasks efficiently and reliably under stringent resource constraints while safely offloading to the cloud when local uncertainty is excessive. The authors propose TSDS, a novel framework that jointly calibrates reasoning termination and uncertainty-aware offloading for the first time. TSDS integrates a lightweight convergence probe to detect reasoning completion, a perplexity-driven offloading mechanism, and a multi-objective Learn-Then-Test (LTT) calibration strategy within a ReAct architecture to enable coordinated edge–cloud decision-making. Experiments on GSM8K, HotpotQA, MBPP, and a home robotics benchmark demonstrate that TSDS reduces per-turn inference computational overhead by 43%–73% compared to offloading-only baselines, while providing theoretical guarantees on task reward and cloud invocation rates.

Abstract

LLM agents following the ReAct paradigm are promising enablers of complex multi-step tasks, including multi-hop question answering, code generation, and control of physical AI systems. Yet, when deployed at the edge, they must tightly manage their reasoning budget while remaining reliable and deferring to a cloud-side model only when local uncertainty is too high to act safely. We propose Think Short, Defer Smart (TSDS), a framework that synergistically integrates a lightweight convergence probe, which halts on-device reasoning once the intended action has stabilized, with a perplexity-based deferral rule that escalates uncertain actions to a cloud-side model. Both mechanisms are jointly calibrated on end-to-end episode trajectories via a multi-objective Learn-Then-Test (LTT) procedure, providing simultaneous finite-sample guarantees on expected episode reward and cloud-call rate. We evaluate TSDS on four ReAct benchmarks spanning arithmetic reasoning (GSM8K), multi-hop question answering (HotpotQA), code generation (MBPP), and multi-step embodied planning (household robot), and compare against thought-calibration-only and calibrated-deferral-only standalone baselines. TSDS reduces per-episode thinking compute by 43%-73% over deferral-only baselines across HotpotQA, MBPP, and the household robot task, while maintaining certified reward and cloud-call rate guarantees.

Problem

• Edge LLM Agents
• Reasoning Budget
• Uncertainty-Aware Deferral
• ReAct Paradigm
• Cloud-Edge Collaboration

Innovation

• edge LLM agents
• calibrated reasoning
• uncertainty-aware deferral
• convergence probe
• Learn-Then-Test