MemSFT: Mitigating Alignment Tax with an External Parametric Memory

2026-07-28

AI Summary

This work addresses the degradation of general capabilities in large language models during domain-specific fine-tuning—a phenomenon often attributed to “alignment tax.” To mitigate this, the authors propose a plug-in parametric memory module that decouples domain knowledge acquisition from the backbone model. The module is trained via behavioral cloning to emulate a non-parametric retriever and is dynamically fused with the backbone’s output during inference through a learned router. This approach achieves, for the first time, parameter-level disentanglement between general-purpose abilities and domain-specific knowledge, enabling memory modules to be reused across different model scales without catastrophic forgetting. Experiments in biomedical, geoscience, and legal domains demonstrate substantial gains in domain performance while preserving near-original general-task proficiency, outperforming standard supervised fine-tuning baselines.

Abstract

Adapting Large Language Models (LLMs) to specialized domains often incurs an alignment tax, as fine-tuning on domain-specific tasks can cause catastrophic forgetting and substantially degrade performance on general tasks. We propose MemSFT, which mitigates the alignment tax by decoupling domain specialization from backbone parameter updates through a plug-and-play parametric memory. The memory is trained to imitate the behavior of a non-parametric retriever operating over domain data, thereby memorizing knowledge and patterns that would otherwise be accessed through retrieval. Once trained on a specific domain, the memory can be reused across LLMs of different sizes. During generation, a learned router dynamically fuses the output distributions of the memory and backbone at each decoding step, allowing domain expertise to be invoked selectively. Across biology, geoscience, and law, evaluations with models ranging from Qwen3-8B to Qwen3-235B-A22B show that MemSFT consistently improves domain performance with negligible degradation in general performance, whereas full SFT suffers severe forgetting on general tasks. Overall, our results demonstrate a practical path to decoupling general model capabilities from domain-specific knowledge at the parameter level, thereby equipping LLMs with new specialized capabilities without compromising their general capabilities.

Problem

• alignment tax
• catastrophic forgetting
• domain adaptation
• large language models
• specialized domains

Innovation

• parametric memory
• alignment tax
• catastrophic forgetting
• domain adaptation
• plug-and-play