From Parameters to Behaviors: A Survey of Model Fusion for Large Language Models

2026-09-16

AI Summary

本文通过定义模型融合并将其分为参数级、表示级和行为级三类，解决了大型语言模型能力整合的问题，并综述了相关度量、基准及应用。

Abstract

Model fusion integrates the capabilities from source models into a single target model. As of June 2026, Hugging Face hosts more than 2M models. This growing pool provides a rich base for model reuse and capability integration. Yet existing surveys often cover only separate parts of this space, and they do not provide a unified definition or a systematic taxonomy. This survey defines model fusion and organizes prior work into three levels: parameter-level, representation-level, and behavior-level fusion. We also review related metrics, benchmarks, and applications, summarize current challenges, and identify future directions. Our goal is to provide a clear map of this area and support future work on model fusion. A comprehensive list of papers about model fusion is available at https://github.com/Baicaihaochi/Awesome-Model-Fusion-Survey.

Problem

• model fusion
• unified definition
• systematic taxonomy
• capability integration

Innovation

• model fusion
• parameter-level fusion
• representation-level fusion
• behavior-level fusion