Combating Knowledge Corruption in Agent Systems: A Byzantine-Tolerant Secure Collaborative RAG Framework

2026-08-04

AI Summary

This work addresses the vulnerability of Retrieval-Augmented Generation (RAG) systems to knowledge pollution attacks, wherein adversaries inject poisoned documents to mislead large language models. To counter this threat, the authors propose SecureCollaRAG, a novel framework that, for the first time, integrates Byzantine fault tolerance into a collaborative RAG architecture. By leveraging dynamic graph neural networks, SecureCollaRAG performs credibility scoring and validation of multi-source knowledge, enabling robust verification even under non-independent and identically distributed (non-IID) data conditions. The approach effectively mitigates stealthy pollution attacks while preserving the integrity of domain-specific knowledge. Both empirical experiments and formal security analysis demonstrate the framework’s efficacy and resilience.

Abstract

While retrieval-augmented generation systems partially address the hallucination issues in large language models, it also introduces new vulnerabilities to knowledge corruption attacks. Adversaries exploit these vulnerabilities by poisoning documents provided by RAG system to manipulate LLM outputs. To counter this threat, we propose SecureCollaRAG, a Byzantine-tolerant collaborative RAG framework leveraging Multi-source Knowledge Validation Mechanism. Our approach enables agent system to securely verify document provenance through dynamic GNN-based credibility scoring, effectively preventing stealthy knowledge corruption attacks while preserving essential domain knowledge integrity. Through extensive evaluations and formal analysis, we demonstrate that SecureCollaRAG maintains robustness against attackers under non-IID data distributions.

Problem

• knowledge corruption
• retrieval-augmented generation
• adversarial attack
• Byzantine tolerance
• document poisoning

Innovation

• Byzantine-tolerant
• Secure Collaborative RAG
• Knowledge Corruption Defense
• Multi-source Knowledge Validation
• GNN-based Credibility Scoring