CoEvoKG: Co-Evolving Knowledge Graphs with Self-Evolving Search Agents

2026-08-03

AI Summary

Existing self-play search agents struggle to reuse knowledge accumulated during successful searches, hindering continual learning and evolution. This work proposes CoEvoKG, a novel framework that uniquely leverages a knowledge graph both as a source for generating verifiable multi-hop tasks and as a persistent memory store for evidence. By jointly training a task generator and a search agent, the framework enables their co-evolution: validated successful search paths are written back into the graph to generate increasingly challenging tasks. Integrating reinforcement learning, multi-hop question generation, dynamic knowledge graph construction and updating, and evidence verification, CoEvoKG achieves average accuracy gains of 10.1–11.6 percentage points over strong baselines across six question-answering benchmarks, significantly outperforming current self-play and reinforcement learning approaches by 2.6–3.7%.

Abstract

Large language models can improve with reinforcement learning for search agents, yet existing self play agents repeatedly generate tasks while discarding the knowledge gained during successful searches. We introduce CoEvoKG, a framework that turns a knowledge graph into both a source of verifiable training tasks and a persistent evidence memory for agent evolution. CoEvoKG jointly trains a task generator and a search agent: the generator creates multihop questions from entity chains sampled from the knowledge graph, while the agent learns from rewards for answer correctness and search trajectories whose entity paths are supported by graph evidence. When a search succeeds, CoEvoKG verifies and deduplicates the retrieved evidence, then writes it back to the corresponding graph nodes and edges. Future rounds reuse this enriched graph for task generation and reward computation, closing the loop between model self evolution and knowledge accumulation. Experiments on six QA benchmarks (NQ, TriviaQA, PopQA, HotpotQA, 2WikiMultiHopQA, and Bamboogle) with three backbone models show that CoEvoKG improves macro average accuracy over the corresponding base models by +11.2, +10.1, and +11.6 points on Qwen2.5-3B-Instruct, Qwen2.5-7B-Instruct, and Llama-3.1-8B-Instruct, respectively. Under matched training budgets, CoEvoKG further improves over competitive self play baselines and RL baselines for search agents by +2.6 to +3.7 macro average points across the three backbones. Code is available at https://github.com/lazzy1225/CoEvoKG.

Problem

• self-play agents
• knowledge accumulation
• search agents
• reinforcement learning
• task generation

Innovation

• Co-Evolving Knowledge Graphs
• Self-Evolving Search Agents
• Reinforcement Learning
• Evidence Memory
• Multihop Question Answering