CPT: Controllable and Editable Design Variations with Language Models

2026-04-05

AI Summary

This work addresses the challenges of achieving visual diversity and high-quality outputs in design processes, which traditionally rely heavily on manual effort and struggle to scale or personalize effectively. To overcome these limitations, the authors propose a design generation approach based on a decoder-only language model (CPT) coupled with a novel Creative Markup Language (CML). CML provides a structured representation of design semantics and styling, enabling the model to learn layout, color, and typography attributes from professional templates. Notably, this is the first method to leverage language models for generating fully editable design variants—rather than pixel-level outputs—and introduces context-aware mechanisms for color and font prediction. Experimental results demonstrate that the generated designs adhere to established design principles while remaining directly editable in mainstream design tools, thereby offering a scalable, semantically consistent, and style-controllable solution for automated design generation.

Abstract

Designing visually diverse and high-quality designs remains a manual, time-consuming process, limiting scalability and personalization in creative workflows. We present a system for generating editable design variations using a decoder-only language model, the Creative Pre-trained Transformer (CPT), trained to predict visual style attributes in design templates. At the core of our approach is a new representation called Creative Markup Language (CML), a compact, machine-learning-friendly format that captures canvas-level structure, page layout, and element-level details (text, images, and vector graphics), including both content and style. We fine-tune CPT on a large corpus of design templates authored by professional designers, enabling it to learn meaningful, context-aware predictions for attributes such as color schemes and font choices. The model produces semantically structured and stylistically coherent outputs, preserving internal consistency across elements. Unlike generative image models, our system yields fully editable design documents rather than pixel-only images, allowing users to iterate and personalize within a design editor. In experiments, our approach generates contextual color and font variations for existing templates and shows promise in adjusting layouts while maintaining design principles.

Problem

• design automation
• editable design
• visual diversity
• personalization
• creative workflows

Innovation

• Creative Pre-trained Transformer
• Creative Markup Language
• editable design generation
• style-aware language model
• design template variation