Automated Optimization Modeling through Expert-Guided Large Language Model Reasoning

2025-08-20

AI Summary

To address the expert dependence, time consumption, and error-proneness of optimization modeling (OM), this paper proposes ORThought—a framework for LLM-driven automated modeling. Methodologically, it introduces an expert-level modeling paradigm to guide chain-of-thought reasoning and establishes LogiOR, the first complex benchmark for logistics optimization; it also systematically corrects labeling errors in existing datasets—reducing mislabeling rates from 42% to <5%. Without fine-tuning or multi-agent systems, ORThought employs a lightweight, standardized inference strategy. Experiments demonstrate that ORThought significantly outperforms prior approaches in modeling accuracy, computational efficiency, and comprehensive evaluation—covering feasibility, structural correctness, and optimality. It thus establishes a reproducible, scalable paradigm for leveraging LLMs in decision-oriented optimization modeling.

Technology Category

• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Search and Optimization: Learning to Search
• Machine Learning: Optimization

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Optimization Modeling (OM) is essential for solving complex decision-making problems. However, the process remains time-consuming and error-prone, heavily relying on domain experts. While Large Language Models (LLMs) show promise in addressing these challenges through their natural language understanding and reasoning capabilities, current approaches face three critical limitations: high benchmark labeling error rates reaching up to 42%, narrow evaluation scope that only considers optimal values, and computational inefficiency due to heavy reliance on multi-agent systems or model fine-tuning. In this work, we first enhance existing datasets through systematic error correction and more comprehensive annotation. Additionally, we introduce LogiOR, a new optimization modeling benchmark from the logistics domain, containing more complex problems with standardized annotations. Furthermore, we present ORThought, a novel framework that leverages expert-level optimization modeling principles through chain-of-thought reasoning to automate the OM process. Through extensive empirical evaluation, we demonstrate that ORThought outperforms existing approaches, including multi-agent frameworks, with particularly significant advantages on complex optimization problems. Finally, we provide a systematic analysis of our method, identifying critical success factors and failure modes, providing valuable insights for future research on LLM-based optimization modeling.

Problem

• Automating optimization modeling to reduce expert dependency
• Addressing high error rates and narrow evaluation in LLM approaches
• Improving computational efficiency for complex decision-making problems

Innovation

• Expert-guided chain-of-thought reasoning framework
• Systematic error correction and comprehensive annotation
• Novel logistics benchmark with complex standardized problems