OptiTree: Hierarchical Thoughts Generation with Tree Search for LLM Optimization Modeling

2025-10-25

AI Summary

Automated optimization modeling in Operations Research (OR) faces challenges due to the inherent mathematical complexity of OR problems and the limited generalizability of fixed-step decomposition approaches. Method: This paper proposes TreeOR, a tree-search-based adaptive modeling framework that constructs dynamic modeling trees via problem classification and integrates hierarchical reasoning to recursively decompose modeling tasks—including variable declaration, constraint generation, and objective formulation—at multiple granularities. Unlike static pipelines, TreeOR supports extensible path search and cross-level reasoning synthesis. Contribution/Results: TreeOR significantly improves logical consistency and completeness of generated models. Evaluated on multiple OR benchmark datasets, it achieves over 10% higher modeling accuracy compared to the state-of-the-art methods. The implementation is publicly available.

Technology Category

• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Search and Optimization: Mixed Discrete/Continuous Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Optimization modeling is one of the most crucial but technical parts of operations research (OR). To automate the modeling process, existing works have leveraged large language models (LLMs), prompting them to break down tasks into steps for generating variables, constraints, and objectives. However, due to the highly complex mathematical structures inherent in OR problems, standard fixed-step decomposition often fails to achieve high performance. To address this challenge, we introduce OptiTree, a novel tree search approach designed to enhance modeling capabilities for complex problems through adaptive problem decomposition into simpler subproblems. Specifically, we develop a modeling tree that organizes a wide range of OR problems based on their hierarchical problem taxonomy and complexity, with each node representing a problem category and containing relevant high-level modeling thoughts. Given a problem to model, we recurrently search the tree to identify a series of simpler subproblems and synthesize the global modeling thoughts by adaptively integrating the hierarchical thoughts. Experiments show that OptiTree significantly improves the modeling accuracy compared to the state-of-the-art, achieving over 10% improvements on the challenging benchmarks. The code is released at https://github.com/MIRALab-USTC/OptiTree/tree/main.

Problem

• Automating complex optimization modeling with adaptive decomposition
• Enhancing LLM capabilities for hierarchical problem solving
• Improving accuracy in operations research mathematical structures

Innovation

• Hierarchical tree search for adaptive problem decomposition
• Organizes OR problems by taxonomy and complexity levels
• Recursively integrates thoughts from simpler subproblems