Robustness and Regularization in Hierarchical Re-Basin

2025-10-10

AI Summary

This work addresses the contradictory phenomenon in Git Re-Basin—where enhanced robustness co-occurs with degraded accuracy in multi-model fusion. We identify substantial performance deterioration in its original implementation and find its reported robustness gains unattainable under standard evaluation protocols. To resolve this, we propose Hierarchical MergeMany: a two-stage, layer-wise fusion framework that first aligns model weight manifolds via Re-Basin and then performs weighted aggregation per layer. This design reveals, for the first time, that Re-Basin induces an implicit regularization effect during multi-model merging—an effect that strengthens with increasing model count. Experiments on CIFAR-10/100 and ImageNet subsets demonstrate that our method significantly improves adversarial robustness (+3.2–5.8%) and input-perturbation robustness, while constraining average accuracy drop to ≤0.4%. It consistently outperforms both standard MergeMany and the original Re-Basin fusion baseline.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

This paper takes a closer look at Git Re-Basin, an interesting new approach to merge trained models. We propose a hierarchical model merging scheme that significantly outperforms the standard MergeMany algorithm. With our new algorithm, we find that Re-Basin induces adversarial and perturbation robustness into the merged models, with the effect becoming stronger the more models participate in the hierarchical merging scheme. However, in our experiments Re-Basin induces a much bigger performance drop than reported by the original authors.

Problem

• Proposes hierarchical merging scheme to improve model merging performance
• Investigates robustness effects induced by hierarchical Re-Basin merging
• Identifies larger performance drop in Re-Basin than originally reported

Innovation

• Hierarchical model merging scheme improves MergeMany algorithm
• Re-Basin induces adversarial robustness in merged models
• Hierarchical merging strengthens robustness with more models