Rethinking the Reliability of Multi-agent System: A Perspective from Byzantine Fault Tolerance

2025-11-13

AI Summary

This paper addresses Byzantine fault tolerance (BFT) reliability in large language model (LLM)-based multi-agent systems (MAS). We propose CP-WBFT, a novel consensus protocol that—uniquely from a BFT perspective—quantifies the reliability of LLM agents. CP-WBFT integrates confidence probing, LLM inference monitoring, and dynamic weighting, leveraging agents’ reflective and discriminative capabilities to achieve robust information flow regulation. It guarantees information consistency under arbitrary complex network topologies. Experiments demonstrate that CP-WBFT maintains high decision accuracy in mathematical reasoning and security assessment tasks even under an extreme 85.7% Byzantine failure rate—substantially outperforming conventional BFT protocols. Our approach establishes a verifiable, scalable fault-tolerant paradigm for trustworthy LLM-augmented MAS, advancing the foundation for reliable collaborative AI.

Technology Category

• Multiagent Systems: Agent Communication
• Natural Language Processing: Safety and Robustness
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Ensuring the reliability of agent architectures and effectively identifying problematic agents when failures occur are crucial challenges in multi-agent systems (MAS). Advances in large language models (LLMs) have established LLM-based agents as a major branch of MAS, enabling major breakthroughs in complex problem solving and world modeling. However, the reliability implications of this shift remain largely unexplored. i.e., whether substituting traditional agents with LLM-based agents can effectively enhance the reliability of MAS. In this work, we investigate and quantify the reliability of LLM-based agents from the perspective of Byzantine fault tolerance. We observe that LLM-based agents demonstrate stronger skepticism when processing erroneous message flows, a characteristic that enables them to outperform traditional agents across different topological structures. Motivated by the results of the pilot experiment, we design CP-WBFT, a confidence probe-based weighted Byzantine Fault Tolerant consensus mechanism to enhance the stability of MAS with different topologies. It capitalizes on the intrinsic reflective and discriminative capabilities of LLMs by employing a probe-based, weighted information flow transmission method to improve the reliability of LLM-based agents. Extensive experiments demonstrate that CP-WBFT achieves superior performance across diverse network topologies under extreme Byzantine conditions (85.7% fault rate). Notably, our approach surpasses traditional methods by attaining remarkable accuracy on various topologies and maintaining strong reliability in both mathematical reasoning and safety assessment tasks.

Problem

• Investigating reliability of LLM-based multi-agent systems under Byzantine faults
• Designing Byzantine fault tolerance mechanism for diverse network topologies
• Enhancing agent reliability using LLMs' reflective and discriminative capabilities

Innovation

• CP-WBFT consensus mechanism enhances MAS reliability
• Weighted Byzantine Fault Tolerance using confidence probes
• Leverages LLMs' reflective capabilities for fault tolerance