Agent-UCT: Upper Confidence Bounds Applied to Trees for Agentic Workflow Optimization with Cost-Awareness

2026-07-27

AI Summary

This work addresses the challenge of optimizing agent workflows—such as RAG pipelines—under limited evaluation budgets by efficiently navigating the discrete combinatorial space of components. Existing methods overlook the inherent compositional structure, leading to redundant computations. To overcome this, the authors propose Agent-UCT, an algorithm that integrates cost-aware Upper Confidence Bound (UCB) tree search with a prefix reuse mechanism within a unified five-dimensional RAGSpace, enabling exploration while reusing previously executed component prefixes. A novel regularization term based on a bipartite prefix reuse graph is introduced to bias the search toward reusable branches, balancing exploration efficiency and execution overhead. The approach is supported by WTB, a testbed featuring cross-framework recomposition, deterministic replay, and content-addressable caching. Experiments on HotpotQA and UltraDomain demonstrate that Agent-UCT discovers configurations surpassing fixed presets, reducing logical search costs by 73.6% and achieving up to 4.2× real-world speedup.

Abstract

Optimizing agentic workflows, such as retrieval-augmented generation (RAG) pipelines, requires navigating a combinatorial space of discrete component choices under tight evaluation budgets. Existing approaches - heuristic search, black-box optimization, and standard tree search methods - do not explicitly exploit the compositional structure of these workflows, leading to redundant computation and inefficient budget allocation. We introduce Agent-UCT (Agent-based Cost-Aware Upper Confidence Bounds Applied to Trees), a tree search algorithm that extends UCT with a reuse-aware regularization term derived from a bipartite prefix reuse graph. Agent-UCT biases selection toward branches that leverage previously materialized configuration prefixes, reducing redundant execution while maintaining effective exploration. Our framework, RAGSpace, unifies heterogeneous RAG components from LongRAG, LightRAG, and Self-RAG into a five-dimensional configuration space, enabling systematic cross-framework recombination. WTB (Workflow Test Bench) provides deterministic replay, content-addressable caching, and transactional consistency, ensuring that intermediate states are materialized once and reused across the search. Experiments on HotpotQA and UltraDomain demonstrate that Agent-UCT identifies configurations with the highest out-of-sample performance among the evaluated fixed framework presets. Under full-pool evaluation, bipartite prefix reuse reduces logical search cost by 73.6% relative to the no-prefix-sharing cost upper bound. Compared with full-pool evaluation, sampling-based evaluation further achieves a 4.2x wall-clock speedup. Agent-UCT, RAGSpace, and WTB together provide a unified framework for cost-aware, reproducible, and compositionally efficient agentic workflow optimization.

Problem

• agentic workflow optimization
• retrieval-augmented generation
• combinatorial optimization
• cost-awareness
• component reuse

Innovation

• Agent-UCT
• cost-aware optimization
• prefix reuse
• compositional workflow search
• RAGSpace