Not All Task Vectors Need Equal Rank: Energy-Proportional Allocation for Model Merging

2026-09-21

AI Summary

本文提出了一种基于奇异值能量结构的任务自适应策略SERA，用于模型合并中非均匀分配秩容量，以优化多任务模型性能。

Abstract

Model merging aims to combine multiple fine-tuned models derived from a common pretrained model into a single multi-task model without additional joint training. Recent spectral merging methods improve over simple weight averaging by exploiting low-rank structures of task-specific updates, but they commonly assign the same rank capacity to every task. This uniform allocation ignores that task vectors can have heterogeneous spectral complexity, causing the shared merging space to be used suboptimally. In this paper, we propose Spectral Energy-proportional Rank Allocation (SERA), a simple task-adaptive strategy that allocates ranks according to the singular-value energy structure of each task vector. By assigning richer spectral capacity to complex or isolated tasks and fewer directions to compact tasks, SERA extends SVD-based model merging from uniform-capacity merging to task-dependent capacity allocation. Experiments under standard vision model merging protocols show that SERA improves multi-task merging performance while preserving the same total rank budget as existing spectral merging methods. Further analysis demonstrates that task-level spectral concentration is closely related to the per-task effect of adaptive rank allocation, providing insight into when and why SERA is effective.

Problem

• model merging
• spectral complexity
• rank allocation

Innovation

• Spectral Energy-proportional Rank Allocation
• SERA
• singular-value energy structure
• task-adaptive strategy
• multi-task model merging