Knowledge is Not Enough: Injecting RL Skills for Continual Adaptation
This work addresses the challenge that large language models, due to frozen parameters, struggle to effectively incorporate new knowledge and enhance reasoning and decision-making capabilities through supervised fine-tuning (SFT) alone. The authors propose the Parametric Skill Transfer (PaST) framework, which leverages the observation that parameter updates induced by SFT and reinforcement learning (RL) are approximately orthogonal. Building on this insight, PaST introduces a modular mechanism that extracts domain-agnostic skill vectors from a source domain, applies lightweight fine-tuning, and linearly injects them into the target model to enable efficient cross-domain skill transfer. This approach establishes a scalable continual adaptation framework, achieving significant performance gains: +9.9 points over the state-of-the-art self-editing SFT baseline on SQuAD, +8.0 points on LooGLE long-context question answering, and an average +10.3-point improvement in zero-shot tool-use success rate on ToolBench, demonstrating strong generalization capabilities.