PerfAgent: Profiler-Guided Iterative Refinement for Repository-Level Code Optimization

2026-07-21

AI Summary

This work addresses the challenges large language model (LLM) agents face in warehouse-scale code optimization—namely, accurately identifying performance bottlenecks, preserving behavioral consistency, and achieving expert-level speedups. The authors propose a novel closed-loop optimization framework that, for the first time, integrates profiler feedback and behavioral validation into the LLM agent’s iterative refinement cycle, overcoming the limitations of prior approaches that rely solely on test pass/fail outcomes or coarse-grained timing. By continuously generating and refining code patches grounded in profiling evidence, the method significantly enhances efficiency while guaranteeing functional correctness. Experiments demonstrate substantial improvements on the GSO and SWE-fficiency-Lite benchmarks, with expert-level optimization patch success rates rising from 19.6% to 39.2% and from 26% to 74%, respectively—surpassing even high-cost oracle best-of-five baselines.

Abstract

Large language model (LLM) agents now perform well on correctness-oriented repository-level tasks, including SWE-Bench issue resolution and feature implementation in real codebases. However, they still struggle with repository-level code optimization, which requires preserving behavior while improving runtime performance. Passing tests is not enough in this setting; a patch must preserve behavior, implement code optimization, and approach expert speedups. Current agents often miss bottlenecks hidden behind abstraction layers and native extensions, stop after shallow speedups, or insufficiently test the code patches that thus may silently break edge cases. We present PerfAgent, a profiler-guided, verifier-in-the-loop workflow that gives an off-the-shelf coding agent the feedback needed to find real hotspots, improve beyond the first passing patch, and use profiler evidence rather than timing alone to decide what to optimize next. On two challenging optimization benchmarks, GSO and SWE-fficiency-Lite, PerfAgent more than doubles the rate of expert-matching patches over OpenHands with GPT-5.1, improving from 19.6% to 39.2% on GSO and from 26% to 74% on SWE-fficiency-Lite. It also surpasses an oracle best-of-five baseline at substantially lower cost, showing that the gains come from better feedback rather than additional test-time sampling.

Problem

• repository-level code optimization
• runtime performance
• behavior preservation
• performance bottlenecks
• code patches

Innovation

• profiler-guided optimization
• LLM agent
• repository-level code optimization
• iterative refinement
• performance verification