From Debate to Equilibrium: Belief-Driven Multi-Agent LLM Reasoning via Bayesian Nash Equilibrium

2025-06-09

AI Summary

Multi-LLM collaborative reasoning often suffers from high computational overhead and lack of convergence guarantees. This paper pioneers modeling multi-agent LLM coordination as an incomplete-information Bayesian game, grounded theoretically in Bayesian Nash Equilibrium (BNE), and proposes ECON—a hierarchical reinforcement learning framework that achieves provable convergence and a tight regret bound without requiring frequent inter-agent communication. ECON integrates probabilistic belief modeling, distributed policy optimization, and Bayesian game analysis, enabling flexible integration of heterogeneous LLMs. Evaluated on six challenging reasoning and planning benchmarks, ECON achieves an average improvement of 11.2% over state-of-the-art collaborative paradigms. The implementation is publicly available.

Technology Category

• Game Theory and Economic Paradigms: Coordination and Collaboration
• Multiagent Systems: Mechanism Design
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Multi-agent frameworks can substantially boost the reasoning power of large language models (LLMs), but they typically incur heavy computational costs and lack convergence guarantees. To overcome these challenges, we recast multi-LLM coordination as an incomplete-information game and seek a Bayesian Nash equilibrium (BNE), in which each agent optimally responds to its probabilistic beliefs about the strategies of others. We introduce Efficient Coordination via Nash Equilibrium (ECON), a hierarchical reinforcement-learning paradigm that marries distributed reasoning with centralized final output. Under ECON, each LLM independently selects responses that maximize its expected reward, conditioned on its beliefs about co-agents, without requiring costly inter-agent exchanges. We mathematically prove that ECON attains a markedly tighter regret bound than non-equilibrium multi-agent schemes. Empirically, ECON outperforms existing multi-LLM approaches by 11.2% on average across six benchmarks spanning complex reasoning and planning tasks. Further experiments demonstrate ECON's ability to flexibly incorporate additional models, confirming its scalability and paving the way toward larger, more powerful multi-LLM ensembles. The code is publicly available at: https://github.com/tmlr-group/ECON.

Problem

• Overcome high computational costs in multi-agent LLM frameworks
• Ensure convergence in multi-agent LLM reasoning without guarantees
• Enhance reasoning power via Bayesian Nash Equilibrium coordination

Innovation

• Bayesian Nash equilibrium for multi-agent coordination
• Hierarchical reinforcement learning with distributed reasoning
• Tighter regret bound than non-equilibrium approaches