UniCanvas: A Diffusion-base Unified Model for Text-in-Image Joint Generation

2026-06-02

AI Summary

Existing unified vision-language models struggle to simultaneously generate high-quality images and coherent text, while diffusion models, though effective for image synthesis, lack the capability to embed legible text meaningfully. This work proposes UniCanvas, the first framework that leverages diffusion mechanisms to jointly generate images and text end-to-end on a unified pixel canvas. By encoding linguistic semantics into drawable visual patterns that serve as embedded textual elements within the image, UniCanvas transcends the conventional paradigm of discrete token-based text generation. Experimental results demonstrate that UniCanvas significantly outperforms current methods on unified multimodal generation tasks, establishing diffusion-driven in-image text embedding as an efficient and promising approach for holistic vision-language synthesis.

Abstract

Recent years have seen remarkable progress in unified vision-language models handling both multimodal understanding and generation within a single architecture. While autoregressive VLMs can reason across modalities, they fail to generate high-quality images. In contrast, diffusion models produce photorealistic visuals yet struggle to generate coherent text, making it challenging to develop a single unified model that can seamlessly handle both visual and text generation. Recent advances suggest that language can be effectively embedded within visual representations, allowing models to reason about textual semantics directly from images. To this end, we propose UniCanvas, a first attempt that unifies diffusion models to generate interleaved multimodal contents through text-in-image generation. Diffusion models naturally capture transformations on a shared pixel canvas, which can be viewed as world models of visual change. Instead of producing discrete text tokens, the model learns to represent language as visual patterns inside images, leveraging its inherent multimodal embedding space. This design allows the model to "draw" text naturally within a single pixel canvas during image synthesis, achieving seamless multimodal generation. Experiments demonstrate that UniCanvas improves performance over previous unified models, positioning text-in-image generation with diffusion models as a promising unified multimodal generation paradigm.

Problem

• unified vision-language models
• multimodal generation
• diffusion models
• text-in-image generation
• coherent text generation

Innovation

• diffusion models
• text-in-image generation
• unified multimodal generation
• visual language modeling
• pixel canvas