MergeSE: Post-Hoc Model Merging for Software Engineering Tasks Without Retraining

2026-08-04

AI Summary

This work addresses the poor generalization of existing domain-specific code models under distribution shifts and the absence of training-free tools for integrating multiple models to support software engineering tasks. The paper introduces MergeSE, the first end-to-end, training-free model merging framework tailored for software engineering classification tasks—such as code clone detection and vulnerability identification. MergeSE integrates five state-of-the-art algorithms, including TIES and DARE-TIES, and provides functionalities for task registration, compatibility diagnosis, deterministic merging, and one-click deployment. It merges two 124-million-parameter models on CPU in under five seconds, recovers 93% of multi-task performance under cross-language and AI-generated code scenarios, achieves up to a fourfold improvement in generalization, and produces results consistent with reference implementations.

Abstract

Fine-tuned code models often behave as domain specialists and can degrade sharply under distribution shift: in our clone-detection setting, a model trained on same-language clones drops 71\% F1 on cross-language clones, while multi-task training falls to 0.151 F1 on unseen AI-generated clones. Our companion study shows that post-hoc model merging can address this fragmentation, achieving 93\% of multi-task performance without training data while generalizing 4$\times$ better to unseen clone types. However, no practical tool exists that lets SE researchers diagnose checkpoint compatibility, merge specialists, validate results on SE benchmarks, and export models for deployment. We present \textbf{MergeSE}, an open-source CLI and web tool for training-free model merging of HuggingFace encoder checkpoints. While motivated by OOD generalization in clone detection, MergeSE supports SE classification workflows more broadly through a built-in registry of nine task types, including vulnerability detection, defect prediction, and code-smell detection. MergeSE provides five operations: \textit{tasks}, \textit{inspect}, \textit{merge}, \textit{evaluate}, and \textit{export}. It supports five merging algorithms, including TIES, DARE-TIES, Wudi, PCB, and averaging; detects cross-task classification-head mismatches; produces seedable deterministic outputs; and includes bundled benchmark samples for smoke-test reproduction. A full merge of two 124M-parameter checkpoints completes in under 5 seconds on CPU. End-to-end validation confirms that MergeSE-produced checkpoints match reference implementations and recover cross-domain performance from domain-specific specialists. The tool is available online at https://mergese.usask.ca, and the development repository is at https://github.com/srlabUsask/MergeSE.

Problem

• model merging
• distribution shift
• code clone detection
• out-of-domain generalization
• software engineering tasks

Innovation

• post-hoc model merging
• out-of-distribution generalization
• software engineering tasks
• training-free model fusion
• checkpoint compatibility