Multi-level Code Optimization via Mixture of Prompts

2026-07-26

AI Summary

This work addresses the limitations of existing code optimization techniques, which struggle to effectively handle dynamic languages and are often confined to single-level transformations, thereby failing to precisely identify performance bottlenecks. To overcome these challenges, we propose Optimo—a multi-level, pattern-aware code optimization framework powered by a Mixture-of-Prompts (MoP) architecture that, for the first time, integrates the mixture-of-experts paradigm into code optimization. Optimo employs differential profiling to pinpoint critical code structures and performs coordinated optimizations across four abstraction levels—from algorithms down to APIs. Experimental results demonstrate that Optimo achieves up to a 57.48% optimization success rate with a 3.97× speedup on human-written code, and a 42.42% success rate with a 13.51× speedup on LLM-generated code, significantly outperforming current baselines on the COFFE and Effibench benchmarks.

Abstract

Runtime efficiency is a critical factor that impacts both software quality and user satisfaction. There are many approaches proposed for code optimization to improve runtime efficiency. Traditional code optimization methods operate on intermediate representations (IRs) during compilation for static languages. They are effective but struggle to handle dynamic languages that do not require compilation. Recently, large language models (LLMs) have been leveraged to directly optimize source code in dynamic languages. However, these methods fail to identify suitable optimization targets and usually conduct incomprehensive single-level optimization. To address these challenges, we propose Optimo, a multi-level LLM-based code optimization approach built on a novel Mixture-of-Prompts (MoP) architecture. In the MoP architecture, Optimo identifies time-critical code structures as performance bottlenecks via differential profiling. These structures are then routed to some optimization strategies, akin to expert models in MoE, each tailored to optimize specific code patterns. Unlike traditional approaches that focus only on statement-level optimizations, Optimo operates at four levels of abstraction, ranging from coarse-grained algorithmic improvements to fine-grained optimizations in API usage. We evaluate Optimo on two code efficiency benchmarks, COFFE and Effibench. Our results demonstrate that Optimo achieves an up to 57.48% opt%, i.e., the percentage of optimized programs that are correct and at least 10% faster than the original programs, and an up to 3.97x speedup when optimizing human-written code, and it consistently outperforms the best baseline by up to 96.51% in terms of opt%. Furthermore, Optimo achieves an up to 42.42% opt% and an up to 13.51x speedup when optimizing LLM-generated code.

Problem

• code optimization
• dynamic languages
• runtime efficiency
• multi-level optimization
• performance bottlenecks

Innovation

• Mixture-of-Prompts
• multi-level optimization
• LLM-based code optimization
• differential profiling
• performance bottleneck