VDAR-Router: Adaptive LLMs Routing via Verbalized Query Difficulty Analysis Retrieval

2026-07-20

AI Summary

This work addresses a critical limitation in existing large language model (LLM) routing strategies—their neglect of intrinsic query difficulty, which often leads to suboptimal model selection. To overcome this, the authors propose a training-free, efficient routing framework that explicitly incorporates query difficulty analysis and difficulty-aware retrieval. The approach leverages an LLM to generate semantic descriptions of query difficulty, retrieves similar historical queries based on these representations, and selects the most suitable model through a reward function that jointly considers performance and computational cost. Evaluated on three benchmark datasets, the method significantly outperforms current baselines, achieving a superior trade-off between inference cost and task performance.

Abstract

Large language models are increasingly used in practical systems, making efficient model selection important for reducing deployment cost. LLM routing has emerged as a practical solution for allocating each input query to an appropriate model under a desired cost-performance trade-off. Existing routing methods often estimate model suitability from the surface semantics or embedding similarity of the input query. However, such methods may ignore the underlying difficulty of a query, leading to suboptimal routing decisions. To address the challenge, we propose VDAR-Router, a difficulty-aware retrieval-based routing framework. For each input query, VDAR-Router first generates an explicit difficulty analysis. It then retrieves historical examples with similar difficulty profiles. Based on the retrieved records, it estimates candidate model suitability and selects the model using a reward function that considers both performance and cost. Experiments on three datasets show that VDAR-Router consistently achieves better cost-performance trade-offs than existing baselines. These results demonstrate the effectiveness of difficulty-aware retrieval for training-free LLM routing. Case studies further show that explicit query analysis helps retrieve more relevant examples and supports more reliable routing decisions.

Problem

• LLM routing
• query difficulty
• cost-performance trade-off
• model selection
• retrieval-based routing

Innovation

• LLM routing
• query difficulty analysis
• retrieval-based routing
• cost-performance trade-off
• training-free