IR3DE: A Linear Router for Large Language Models

2026-06-04

AI Summary

Existing large language model (LLM) routing approaches suffer from limitations in cost efficiency or training overhead. This work proposes a lightweight linear routing mechanism based on ridge regression, which, to the best of our knowledge, is the first to apply linear models to dynamic selection among multi-domain expert LLMs. By leveraging input features derived from causal language modeling and reasoning tasks, the method enables low-overhead, highly generalizable routing decisions and supports dynamic addition or removal of expert models without retraining the router. Experimental results demonstrate that the approach matches baseline performance across two task categories and significantly outperforms existing methods in reasoning tasks, achieving 98.4% normalized performance.

Abstract

Foundational Large Language Models (LLMs) demonstrate proficiency on a wide range of general tasks, and achieve remarkable results on various specialized tasks via domain-expert LLMs. With the ever-growing list of available LLMs, inference routers are being proposed to select the most appropriate LLM for each prompt. However, existing routing methods either optimize cost across weak-to-strong generalist LLMs or require substantial training to support domain-expertise routing. In this paper, we propose IR3DE, a Ridge Regression-based Router for Domain Experts that provides cheap and fast routing decisions for each prompt. We evaluate IR3DE in two Causal Language Modeling (CLM) settings where the tasks are next-token prediction for all domains, and one reasoning setting where each domain has its own distinct reasoning task. Despite being a linear router, IR3DE achieves performance comparable to the other baselines in both CLM settings, and surpassing them in the reasoning setting, with a normalized performance of 98.4%. Moreover, IR3DE enables the addition or removal of new domain experts without requiring the router to be retrained from scratch, allowing a dynamic set of LLMs to be served with minimal disruption to the router itself. Our code is available at: github.com/gensyn-ai/IR3DE.

Problem

• inference routing
• domain-expert LLMs
• model selection
• dynamic model set
• large language models

Innovation

• Ridge Regression
• LLM Routing
• Domain Experts
• Linear Router
• Dynamic Model Selection