DHCG: Dynamic Construction of Hierarchical Collaboration Graphs for LLM-Based Multi-Agent Reasoning

2026-10-06

AI Summary

This study addresses the limitations of existing LLM-based multi-agent systems in dynamic orchestration, specifically constrained combinatorics, dependency misalignment, and rigid scalability. To overcome these challenges, this work reformulates multi-agent collaboration as a Partially Observable Markov Decision Process (POMDP) and proposes an execution feedback-driven mechanism for progressively constructing hierarchical collaboration graphs. Furthermore, an action-aware preference optimization strategy is introduced to enhance the planner's decision-making efficacy. Experimental results demonstrate that the proposed approach achieves state-of-the-art performance on benchmarks such as code generation, outperforming single-agent baselines by 13.06 points and significantly surpassing existing static and dynamic multi-agent methods.

Abstract

LLM-based multi-agent systems (MAS) have demonstrated strong capabilities in solving complex problems across diverse domains. Recently, the dynamic orchestration of agent systems has become an important research direction. However, existing methods suffer from limited composition, misaligned dependencies, and inflexible scale, restricting their ability to adapt to reasoning requirements during execution. To address these limitations, we reframe MAS design as a partially observable Markov decision process, in which both the composition and scale of the MAS are dynamically determined. We propose DHCG, a novel framework that coordinates three modules (Planner, Worker, and Generator) to progressively construct a dynamic hierarchical collaboration graph from scratch based on the query and evolving execution feedback. At each step, guided by feedback, the Planner generates a set of distinct and complementary roles tailored to the current reasoning needs and selectively routes relevant information to each role. It can also finalize the hierarchical collaboration graph early or progressively expand it when additional reasoning is required. We further introduce action-aware preference optimization to train the Planner to make more effective decisions when constructing hierarchical collaboration graphs. We systematically evaluate DHCG across code generation, mathematical reasoning, and domain-specific reasoning benchmarks. DHCG achieves state-of-the-art average performance among the compared methods, improving over the single-agent baseline by 13.06 points and outperforming both static and dynamic MAS baselines by 2.77-8.02 points. Additional experiments further demonstrate its generalization across different Planner backbones and unseen Worker models.

Problem

• Multi-Agent Systems
• Dynamic Orchestration
• Hierarchical Collaboration
• LLM Reasoning
• Adaptive Composition

Innovation

• Hierarchical Collaboration Graphs
• Multi-Agent Reasoning
• Partially Observable Markov Decision Process
• Action-aware Preference Optimization
• Dynamic Orchestration