One Rewrite to Fix Them All? Type-Aware Repair Allocation for Text-to-Image Prompt Optimization

2026-07-21

AI Summary

Text-to-image generation often suffers from semantic errors—such as counting mistakes, attribute misalignment, ambiguous relationships, or illegible text—due to imperfect prompt understanding. This work proposes TARA, a training-free framework that formulates prompt rectification as atomic operations: leveraging a type-aware diagnosis and assignment mechanism, it routes distinct error types to specialized repair operators and integrates them into a unified executable prompt via a semantic repair gating strategy. Without requiring model retraining, TARA achieves precise corrections without semantic degradation. Evaluated on the DSG and TIFA benchmarks, the method improves semantic accuracy by up to 5.6 points while maintaining image fidelity, with each prompt processed in only 16 seconds.

Abstract

Text-to-image (T2I) generators often fail to follow their prompts faithfully, producing wrong counts, swapped attributes, ambiguous relations, and illegible text. Prompt optimization repairs such failures by rewriting the user prompt, requiring no generator retraining, and has yielded promising results. However, existing optimizers absorb heterogeneous failures into one uniform prompt expansion, even though each calls for different repair language. We formulate semantic prompt optimization as atomic repair allocation: each failed proposition is routed to a type-conditioned repair operator before the resulting local constraints are compiled into one executable prompt. We instantiate this formulation in the training-free Type-Aware Repair Allocation (TARA) framework, which separates diagnosis, allocation, compilation, and a semantic repair gate, an accept-or-revert controller over exactly one prescribed repair that prevents semantic regressions. Extensive experiments on DSG and TIFA across four frozen generators demonstrate that TARA achieves the best semantic accuracy in all eight benchmark-generator cells, improving over VisualPrompter by 5.6 and 2.6 points on DSG and TIFA, respectively, while maintaining image quality and running fastest in our matched local setting at 16.0 seconds versus 20.0 seconds per prompt.

Problem

• text-to-image generation
• prompt optimization
• semantic accuracy
• repair allocation
• type-aware

Innovation

• type-aware repair
• prompt optimization
• text-to-image generation
• semantic accuracy
• repair allocation