LARA: Lightweight Adapters in the Residual Stream for Composable Adaptation and Alignment

2026-07-25

AI Summary

This work addresses the challenge of efficiently adapting and aligning large language models to multiple tasks without modifying their pretrained weights. The authors propose LARA, a lightweight adaptation method that freezes the backbone model and injects low-rank correction signals into the residual stream. LARA introduces token-level dynamic routing within the residual stream for the first time, enabling concurrent hosting and on-demand composition of multiple behavioral modules. It further incorporates a tunable interpolation coefficient γ to enable smooth control over behavior blending. With only 33 MB of additional overhead, LARA supports the simultaneous deployment of seven distinct behaviors on a 1.5B-parameter model, achieving performance comparable to LoRA on code fine-tuning and DPO tasks while significantly enhancing deployment flexibility and resource efficiency.

Abstract

We present LARA (Lightweight Additive Residual Adaptation), a method for efficient adaptation that operates in the residual stream of a frozen model rather than in its weights. Where LoRA adds an update of low rank to weight matrices, LARA reads the hidden state at a small set of layers and adds a correction of low rank back to the residual stream, leaving all base weights untouched. On a code fine-tuning task and on preference optimization (DPO), LARA matches LoRA at equal parameter counts. Because adaptation is a frozen base plus a residual, LARA exposes a scale γ, applied at inference, that interpolates smoothly between base and adapted behavior, a form of graded control that adaptation in weight space does not offer. Finally, because each behavior is a small residual module over a shared frozen base, many behaviors can be held resident at once and routed automatically per token. We place seven behaviors, six fine-tuned and one optimized for preference, on one frozen 1.5B model for roughly 33 MB of overhead, against one full model for each behavior. Because the base is untouched, behaviors are trained separately and selected per token rather than loaded on demand, which suits hosting many behaviors, and adding new ones, on a single model on a device.

Problem

• model adaptation
• residual stream
• composable adaptation
• parameter-efficient fine-tuning
• multi-behavior modeling

Innovation

• LARA
• residual stream
• parameter-efficient adaptation
• composable models
• token-level routing