π€ AI Summary
This study addresses the inefficiency and suboptimal performance of manually crafted static rules in AI coding agents by proposing RuleEvolve, a self-evolving framework. This work pioneers an automatic evolution mechanism for coding rules, leveraging large language model-driven mutators to generate rule variants and employing a multi-agent evaluation module to iteratively refine a candidate rule pool. By eliminating reliance on human expertise, the framework achieves dynamic self-optimization. Extensive evaluations across multiple benchmarks demonstrate that RuleEvolve significantly outperforms both handcrafted rules and existing prompt optimization baselines in terms of functional correctness, code length, and generation cost.
π Abstract
The performance of AI coding agents is highly dependent on their underlying coding rules. However, existing coding rules are typically hand-crafted and fixed, making the process labor-intensive and often suboptimal. In this work, we propose RuleEvolve, a self-evolving framework for coding rules. RuleEvolve maintains a pool of candidate coding rules and iteratively improves them. In each iteration, it employs an LLM-powered mutator module to generate variants from existing candidates, and then uses a judge module to evaluate these variants and update the pool with the best-performing ones. Extensive evaluations across two coding-agent frameworks, four backbone LLMs, and three benchmarks demonstrate that RuleEvolve outperforms both manual engineering and existing prompt optimization baselines in terms of functional correctness of the generated code, code length, and/or generation cost (e.g., tokens used).