Algorithm Generation via Creative Ideation

2025-10-04

AI Summary

Large language models (LLMs) struggle with algorithmic innovation in systems design due to discontinuous solution spaces, often converging on generic heuristics and failing to achieve necessary creative leaps. Method: We propose MetaMuse, a framework integrating three synergistic mechanisms: (1) performance-space–based self-reflection to quantitatively balance diversity and performance; (2) external stimulus guidance to steer targeted creative generation; and (3) executable waypoint-guided chain-of-thought reasoning—replacing unstructured free association—to guarantee algorithmic structural correctness and implementability. Contribution/Results: Evaluated on two canonical systems problems—cache replacement and online bin packing—MetaMuse autonomously generates algorithms that reduce cache miss rates by 35.76% and container redundancy by 30.93%, significantly outperforming both human-tuned baselines and existing LLM-driven approaches. This work marks the first demonstration of high-performance, autonomous algorithm synthesis for discontinuous solution spaces in systems optimization.

Technology Category

• Search and Optimization: Metareasoning and Metaheuristics
• Multiagent Systems: Mechanism Design
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Designing system algorithms remains challenging, where the discontinuous nature of the solution space often forces system engineers to rely on generic heuristics at the expense of performance. We study whether LLMs can practically drive algorithm generation, and find that they are biased towards well-known generic designs, rather than making the creative leaps needed to navigate the discontinuous solution space. To address this limitation, we introduce MetaMuse, a framework for creative ideation built on three self-reflection principles: (1) quantifying solution diversity and usefulness in measurable performance space, rather than abstract idea space, (2) steering ideation through external stimuli, rather than internal randomness, and (3) constructing executable solutions using waypoint reasoning, rather than free-form chain-of-thought. Extensive evaluation shows that MetaMuse can generate high-performing solutions for two critical problems at a global cloud provider: cache replacement (reducing cache misses by up to 35.76%) and online bin packing (reducing bin usage by up to 30.93%).

Problem

• Generating creative algorithms beyond generic heuristics
• Overcoming LLM bias towards conventional design solutions
• Navigating discontinuous solution spaces for system optimization

Innovation

• Quantifying diversity in measurable performance space
• Steering ideation through external stimuli
• Constructing solutions using waypoint reasoning