Scaling Properties of Text Conditioning in Visual Generation

2026-07-31

AI Summary

This work addresses the lack of systematic understanding regarding how the natural language structure in textual prompts influences the training efficacy of vision generation models. It reveals, for the first time, linear and power-law scaling relationships between diffusion loss and the amount of structured language in prompts, leading to a novel paradigm for enhancing both “diffusability” and “promptability.” The authors introduce a white-box language likelihood metric (GPG) and a black-box attribute-based metric (ED), leveraging semantic-geometric annotations to construct structured prompts. A prompter is trained via supervised fine-tuning, cold-start initialization, and online distillation with a verifier gating mechanism. The resulting method outperforms all existing open-source models across nearly all compositional, reasoning, and world-knowledge benchmarks, and matches or exceeds the performance of the strongest closed-source models on most evaluations.

Abstract

We study empirical scaling properties for text conditioning in visual generation. Such properties have rarely been measured because diffusion loss does not scale with the number of tokens in natural-language prompts. Surprisingly, we find that the converged diffusion loss scales with the amount of structured language in the prompt. To quantify structured language, we adapt two complementary measures: a white-box likelihood metric (GPG) and a black-box attribute metric (ED). Across controlled training runs, the converged diffusion loss decreases approximately linearly with GPG and follows a power law with ED. Guided by these scaling properties, we improve \emph{diffusability} by constructing structured prompts with semantic and geometric annotations derived from images, and improve \emph{promptability} by training a prompter through supervised fine-tuning, cold-start, and verifier-gated on-policy distillation. The resulting system outperforms all evaluated open-weight models on nearly every compositional, reasoning, and world-knowledge benchmark, while matching or surpassing the strongest closed-weight models on most evaluations.

Problem

• text conditioning
• visual generation
• scaling properties
• diffusion loss
• structured language

Innovation

• structured prompting
• diffusability
• promptability
• scaling laws
• text-to-image generation