🤖 AI Summary
Existing LLM orchestration scripts suffer from insufficient modularity and limited parallelization capabilities. To address these issues, this paper proposes an abstract framework grounded in algebraic effects and composable effect handlers, which decouples side effects—such as LLM invocations, I/O operations, and concurrency—into replaceable, composable effect interfaces. This design enforces a strict separation between workflow logic and execution details, preserving code clarity and maintainability while natively enabling fine-grained parallel scheduling and optimization. Evaluated on Tree-of-Thoughts reasoning tasks, the framework achieves a 10× end-to-end performance improvement over baseline approaches. Results demonstrate that the method effectively enhances execution efficiency without compromising modularity, confirming its validity and broad applicability across LLM-driven workflows.
📝 Abstract
Implementing LLM-integrated scripts introduces challenges in modularity and performance, as scripts are often coupled to specific LLM implementations and fail to exploit parallelization opportunities. This paper proposes using composable effect handling to separate workflow logic from effectful operations, such as LLM calls, I/O, and concurrency, enabling modularity without sacrificing the opportunity for performance optimization. By treating these operations as abstract interfaces and discharging them via effect handlers, this paper shows that scripts can achieve significant speedups (e.g., 10$ imes$ in a Tree-of-Thoughts case study) without compromising modularity. This paper aims to promote composable effect handling as a programming style for LLM scripting.