The impact of multi-agent debate protocols on debate quality: a controlled case study

2026-03-28

AI Summary

This study addresses the challenge of disentangling the effects of protocol design from intrinsic model capabilities in multi-agent debate settings. Through controlled experiments in a macroeconomic scenario, the authors compare four interaction protocols: Within-Round (WR), Cross-Round (CR), a novel Rank-Adaptive Cross-Round (RA-CR), and a non-interactive baseline. The RA-CR protocol innovatively incorporates an external evaluator to dynamically adjust speaking order and progressively mute the weakest-performing agent each round, thereby significantly accelerating consensus convergence. Results demonstrate that RA-CR achieves the highest consensus efficiency, WR exhibits the greatest peer citation rate, and the non-interactive baseline preserves the highest argument diversity, collectively revealing an inherent trade-off between interactivity and convergence in multi-agent deliberation.

Technology Category

Multiagent Systems: Agreement, Argumentation & NegotiationNatural Language Processing: Discourse, Pragmatics & Argument MiningGame Theory and Economic Paradigms: Mechanism Design

Application Category

Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environmentsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingResponsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

In multi-agent debate (MAD) systems, performance gains are often reported; however, because the debate protocol (e.g., number of agents, rounds, and aggregation rule) is typically held fixed while model-related factors vary, it is difficult to disentangle protocol effects from model effects. To isolate these effects, we compare three main protocols, Within-Round (WR; agents see only current-round contributions), Cross-Round (CR; full prior-round context), and novel Rank-Adaptive Cross-Round (RA-CR; dynamically reorders agents and silences one per round via an external judge model), against a No-Interaction baseline (NI; independent responses without peer visibility). In a controlled macroeconomic case study (20 diverse events, five random seeds, matched prompts/decoding), RA-CR achieves faster convergence than CR, WR shows higher peer-referencing, and NI maximizes Argument Diversity (unaffected across the main protocols). These results reveal a trade-off between interaction (peer-referencing rate) and convergence (consensus formation), confirming protocol design matters. When consensus is prioritized, RA-CR outperforms the others.

Problem

• multi-agent debate
• debate protocol
• protocol effects
• debate quality
• consensus formation

Innovation

• multi-agent debate
• debate protocol
• Rank-Adaptive Cross-Round
• consensus convergence
• controlled case study