BACAM: Behavior-Aware Continual Agent Merging for Multi-Turn Interaction

2026-10-04

AI Summary

This study addresses the limitations of parameter- or feature-space merging methods for continual agent integration, which fail to guarantee behavioral inheritance and are susceptible to interference from new expert updates on existing interactive capabilities. We propose BACAM, a method that learns parameter-level merging gates from trajectories via expert-guided behavioral supervision. By incorporating task-level stability-plasticity control and tensor-level conflict-aware update budgets, BACM enables the continual integration and stable retention of multi-turn interactive abilities. Furthermore, a weight folding strategy is employed to ensure zero additional inference overhead. Evaluated across four interactive tasks, including web shopping, BACAM achieves an average success rate of 62.82%, outperforming the strongest baseline by 21.69 percentage points. The code has been made publicly available.

Abstract

Model merging offers a way to integrate the capabilities of specialized experts, but existing agent merging methods typically require all of them to be available at once. We study continual agent merging, which integrates incoming experts sequentially without retaining previously merged experts. Yet merging in parameter space or feature subspaces does not ensure that the merged model acquires an incoming expert's behavior on interaction trajectories. Moreover, updates toward a new expert can disrupt the merged model's previously integrated interactive behavior. Therefore, we propose Behavior-Aware Continual Agent Merging (BACAM), which learns parameter-wise merging gates from candidate-generated trajectories using expert-guided behavioral supervision. Task-level stability-plasticity control and tensor-level conflict-aware update budgets limit interference with existing capabilities while allowing new ones to be acquired. The learned gates are folded into the model weights without additional inference-time parameters. Across four interactive tasks - web shopping, tool use, information retrieval, and embodied interaction - BACAM achieves an average success rate of 62.82%, exceeding the strongest evaluated merging baseline by 21.69 percentage points. Our code is publicly available at https://github.com/shuaitongli/BACAM.

Problem

• Continual Agent Merging
• Model Merging
• Multi-Turn Interaction
• Behavioral Alignment
• Catastrophic Forgetting

Innovation

• Continual Agent Merging
• Behavior-Aware Supervision
• Parameter-wise Merging Gates
• Stability-Plasticity Control
• Multi-Turn Interaction