AutoRef: Harness Optimization for Agentic Multi-Reference Image Generation

2026-09-28

AI Summary

This study addresses the high costs and unstable performance associated with manually designed execution frameworks for multi-reference image generation. To overcome these limitations, this work proposes an automated optimization system driven by a large language model-based coding agent. By keeping the foundation model frozen, the method decouples feedback and selection tasks and incorporates a beam search mechanism to guide the agent in iteratively rewriting execution code, thereby automatically discovering high-performing generation frameworks. Experimental results demonstrate that the proposed system elevates the score of the open-source FLUX model on the MultiBanana benchmark to 7.37, achieving performance comparable to proprietary models while exhibiting strong generalization capabilities across diverse configurations.

Abstract

Recent image generation models can take multiple reference images as input and combine them into a new image. However, multi-reference image generation remains challenging: models may omit or duplicate subjects from the references, or produce images in which multiple subjects appear unnaturally pasted. Recent work has proposed image generation agents that combine image generation models, reasoning models, and a harness, which is an executable program that specifies how reference images are interpreted, how generation is performed, how outputs are diagnosed, and how the final image is selected. In multi-reference generation, however, references play different roles and outputs must satisfy many criteria at once, such as fidelity to each reference and the naturalness of the whole image, so many parts of the harness could be improved, from how references are processed to how outputs are diagnosed. This makes it hard to predict which changes will improve performance and by how much, and good harnesses difficult to design by hand; indeed, human-written harnesses vary widely in performance. We therefore propose AutoRef, which optimizes the harness automatically while keeping both models frozen: a coding agent iteratively rewrites the harness code. AutoRef separates the tasks whose feedback informs proposals from the tasks used to select candidates, and continues the search from a beam of the top-ranked harnesses on the selection tasks. Using this procedure, we discover AutoRef-Harness, which improves the open-weight FLUX.2 [klein] 4B from 5.72 to 7.37 on held-out four-reference tasks of the MultiBanana benchmark, matching or exceeding proprietary models including Nano Banana Pro and GPT-Image-1.5. Without re-optimization, the same harness also improves results when the generator, number of references, benchmark, evaluator, or reasoning model differs from those used in the search.

Problem

• multi-reference image generation
• harness optimization
• image generation agent
• automated code optimization

Innovation

• Multi-Reference Image Generation
• Harness Optimization
• Agentic Workflow
• Beam Search
• Code Rewriting Agent