Sharpness-aware Model Merging with Salience Recovery for LLM-based Cross-Domain Sequential Recommendation

2026-07-28

AI Summary

This work addresses the performance saturation and knowledge conflicts in cross-domain sequential recommendation caused by parameter misalignment and statistical homogenization in large language models. To mitigate these issues, the authors propose a协同 optimization framework that constructs an interference-free fused parameter space through a sharpness-aware geometric alignment mechanism and recovers critical target-domain features via a preference saliency activation strategy. This approach effectively alleviates cross-domain knowledge conflicts and overcomes the performance bottleneck inherent in multi-domain fusion. Extensive experiments demonstrate that the proposed model significantly outperforms state-of-the-art baselines in both dual-domain and multi-domain settings, confirming its effectiveness and strong generalization capability.

Abstract

LLM-based Cross-Domain Sequential Recommendation (CDSR) leverages LLMs to enhance target performance via deep semantic reasoning, alleviating the dependency on overlapping users. Among LLM-based paradigms, model merging is particularly promising for multi-domain scenarios due to its superior scalability and flexibility in integrating diverse knowledge sources. However, our empirical investigations reveal two critical bottlenecks: (1) cross-domain knowledge conflict; and (2) performance saturation in multi-domain fusion. Our analysis attributes these phenomena to parameter-level misalignment and statistical homogenization during the merging process. To address these bottlenecks, we propose SharpRec, Sharpness-aware Model Merging with Salience Recovery for LLM-based CDSR, a framework designed to lift the performance upper bound of merged models. SharpRec incorporates two synergistic modules: Sharpness-aware Geometric Alignment to establish a stable geometric foundation for interference-free fusion; and Preference Salience Activation to effectively recover the distinctive features essential for bolstering target domain performance. Extensive experiments in both dual-domain and multi-domain scenarios demonstrate that SharpRec consistently outperforms state-of-the-art baselines.

Problem

• Cross-Domain Sequential Recommendation
• Model Merging
• Knowledge Conflict
• Performance Saturation
• LLM-based Recommendation

Innovation

• Sharpness-aware
• Model Merging
• Salience Recovery
• Cross-Domain Sequential Recommendation
• Geometric Alignment