STEDiff: Strengthening Text Embedding for Text-to-Image Alignment in Diffusion Model

2026-06-09

AI Summary

Existing text-to-image generation models often struggle to accurately align semantics with complex prompts, frequently omitting objects or misassigning attributes. This work proposes a training-free optimization method in the text embedding space that enhances semantic fidelity by reconstructing the [EOT] token to strengthen clause-level semantics and introducing a semantic enhancement loss to impose spatial constraints. Without requiring fine-tuning or layout priors, the approach achieves precise semantic enhancement directly within the textual embedding space for the first time. Evaluated on the T2I-CompBench benchmark, the method significantly outperforms current state-of-the-art techniques, simultaneously improving both generation completeness and semantic alignment in complex compositional scenarios.

Abstract

Although pretrained text-to-image (T2I) generation models can produce high-quality images, they often fail to faithfully reflect the semantic intent of complex prompts due to stochastic noise and inherent model limitations. This issue frequently manifests as the model overlooking specific objects or failing to correctly bind attributes to their corresponding entities, a challenge referred to as semantic alignment. Unlike existing approaches that rely on computationally expensive fine-tuning or labor-intensive layout priors, we propose STEDiff, a training-free method designed to enhance semantic representations directly within the text-embedding space. Specifically, we introduce a method that primarily leverages the [EOT] token to strengthen the relevant semantics of sub-sentences and then replaces the corresponding tokens in the original prompt. Furthermore, a novel semantic enhancement loss is incorporated to enforce spatial constraints, ensuring that the semantics of each entity are precisely mapped to their respective image regions. Extensive quantitative and qualitative evaluations on the T2I-CompBench demonstrate that our method notably improves semantic consistency and generation integrity in complex scenarios.

Problem

• text-to-image generation
• semantic alignment
• complex prompts
• attribute binding
• semantic consistency

Innovation

• text-to-image alignment
• semantic enhancement
• diffusion model
• training-free method
• text embedding