Tree of Agents: Improving Long-Context Capabilities of Large Language Models through Multi-Perspective Reasoning

2025-09-08

AI Summary

Large language models (LLMs) suffer from “middle information loss” in long-context tasks—critical content in input mid-segments is frequently overlooked, inducing positional bias and hallucination. Method: We propose TreeMA, a tree-structured multi-agent reasoning framework that partitions long inputs for parallel processing, enables dynamic inter-agent collaboration and multi-perspective reasoning via hierarchical tree paths, and incorporates prefix hashing caching and adaptive pruning to enhance computational efficiency. Contribution/Results: TreeMA mitigates attention dilution and redundant computation without sacrificing key information. Evaluated on LLaMA3.1-8B, it achieves state-of-the-art performance across multiple long-context benchmarks—matching or exceeding commercial models such as Gemini 1.5 Pro—while incurring comparable API costs.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Multiagent Systems: Multiagent Learning

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Large language models (LLMs) face persistent challenges when handling long-context tasks, most notably the lost in the middle issue, where information located in the middle of a long input tends to be underutilized. Some existing methods that reduce input have the risk of discarding key information, while others that extend context windows often lead to attention dispersion. To address these limitations, we propose Tree of Agents (TOA), a multi-agent reasoning framework that segments the input into chunks processed by independent agents. Each agent generates its local cognition, then agents dynamically exchange information for collaborative reasoning along tree-structured paths. TOA enables agents to probe different reasoning orders for multi-perspective understanding, effectively mitigating position bias and reducing hallucinations. To improve processing efficiency, we incorporate prefix-hash caching and adaptive pruning strategies, achieving significant performance improvements with comparable API overhead. Experiments show that TOA, powered by compact LLaMA3.1-8B, significantly outperforms multiple baselines and demonstrates comparable performance to the latest and much larger commercial models, such as Gemini1.5-pro, on various long-context tasks. Code is available at https://github.com/Aireduce952/Tree-of-Agents.

Problem

• Addresses lost-in-the-middle issue in long-context LLMs
• Mitigates position bias and reduces hallucination risks
• Improves efficiency while maintaining comparable API overhead

Innovation

• Multi-agent framework segments input chunks
• Tree-structured paths enable dynamic information exchange
• Prefix-hash caching and adaptive pruning improve efficiency