HDGlyph: A Hierarchical Disentangled Glyph-Based Framework for Long-Tail Text Rendering in Diffusion Models

2025-05-10

AI Summary

Diffusion models suffer from inaccurate rendering of long-tail text—such as small-font, unseen, or multilingual characters—in text-to-image generation. To address this, we propose a hierarchical disentanglement framework that decouples text coloring from background generation, enabling joint optimization of generalizability and long-tail robustness. Our method introduces three key innovations: (1) a multilingual GlyphNet architecture with glyph-aware design; (2) a glyph-perceptual loss to enhance character-level fidelity; and (3) the first noise-disentangled classifier-free guidance (CFG) and a two-stage latent-space disentangled rendering mechanism. Extensive experiments demonstrate significant improvements: text rendering accuracy increases by 5.08% on English and 11.7% on Chinese benchmarks, respectively, while image quality and long-tail generation robustness are substantially enhanced.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Visual text rendering, which aims to accurately integrate specified textual content within generated images, is critical for various applications such as commercial design. Despite recent advances, current methods struggle with long-tail text cases, particularly when handling unseen or small-sized text. In this work, we propose a novel Hierarchical Disentangled Glyph-Based framework (HDGlyph) that hierarchically decouples text generation from non-text visual synthesis, enabling joint optimization of both common and long-tail text rendering. At the training stage, HDGlyph disentangles pixel-level representations via the Multi-Linguistic GlyphNet and the Glyph-Aware Perceptual Loss, ensuring robust rendering even for unseen characters. At inference time, HDGlyph applies Noise-Disentangled Classifier-Free Guidance and Latent-Disentangled Two-Stage Rendering (LD-TSR) scheme, which refines both background and small-sized text. Extensive evaluations show our model consistently outperforms others, with 5.08% and 11.7% accuracy gains in English and Chinese text rendering while maintaining high image quality. It also excels in long-tail scenarios with strong accuracy and visual performance.

Problem

• Improves long-tail text rendering in diffusion models
• Handles unseen or small-sized text effectively
• Ensures robust multilingual character rendering

Innovation

• Hierarchical decoupling of text and visual synthesis
• Multi-Linguistic GlyphNet for robust character rendering
• Noise-Disentangled Classifier-Free Guidance for refinement