Problem
Research questions and friction points this paper is trying to address.
LoRA composition
training-free routing
expert coordination
modular AI systems
JevSoup: System-One Routing for Training-Free LoRA Composition
🤖 AI Summary
This study addresses the limitations of existing LoRA composition routing methods, which typically rely on auxiliary data, additional training, or autoregressive decoding. To overcome these constraints, this work proposes a training-free framework that decouples System One fast routing from System Two execution. Specifically, the method performs structured probabilistic routing to select two experts based solely on the input and expert descriptions, subsequently achieving lossless merging of expert updates via projection onto the orthogonal complement space. As the first training-free routing mechanism of its kind, the proposed approach demonstrates significant efficacy on the PorTAL task using the Qwen3 model, improving macro-accuracy and micro-accuracy by 1.19% and 1.21%, respectively. These results confirm that the framework enables both efficient and precise LoRA composition without incurring additional training overhead.
📝 Abstract
Building adaptable AI systems requires effective coordination of specialized capabilities across diverse tasks. Low-rank adaptation (LoRA) enables modular expertise, but existing routing approaches may require auxiliary data, additional training, or autoregressive decoding. We propose JevSoup, a training-free framework separating System One expert routing from System Two execution. Using only the input and expert descriptions, Jev selects two experts through structured probabilities. JevSoup retains the leading expert's update, projects the second onto the orthogonal complement of the first update's row space, and combines them with equal weights. Across 14 PorTAL tasks and three Qwen3 scales, JepSoup achieves absolute gains of up to 1.19\% in task-macro and 1.21\% in sample-micro accuracy over the strongest evaluated external baselines. Our code is available at https://github.com/Leowang980/JevSoup.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Training-Free LoRA Composition
System-One Routing
Orthogonal Projection
Expert Routing
Modular Expertise
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
