FontCraft: Multimodal Font Design Using Interactive Bayesian Optimization

2025-02-17

AI Summary

Existing automatic font generation methods require users to manually sketch stylized characters, imposing prohibitively high barriers for non-expert users. This paper introduces FontCraft—the first interactive font design system tailored for non-experts, capable of generating high-quality, stylistically consistent OpenType fonts without requiring any pre-drawn glyphs. Its key contributions are: (1) multimodal style conditioning via images or text prompts, integrated with human preference-driven Bayesian optimization; (2) the first reversible-selection preference Bayesian optimization mechanism; (3) a variational autoencoder-based font latent space with iterative glyph-style propagation; and (4) a history-aware interface with fused multimodal embeddings. A user study demonstrates that FontCraft significantly improves design efficiency and user satisfaction among non-experts, enabling end-to-end font generation within minutes.

Technology Category

• Humans and AI: Intelligent User Interfaces
• Search and Optimization: Non-convex Optimization
• Natural Language Processing: Generation

Application Category

• Economics, Online Markets and Human Computation: Economic aspects and information design of Web, GenAI, and cloud computing
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Creating new fonts requires a lot of human effort and professional typographic knowledge. Despite the rapid advancements of automatic font generation models, existing methods require users to prepare pre-designed characters with target styles using font-editing software, which poses a problem for non-expert users. To address this limitation, we propose FontCraft, a system that enables font generation without relying on pre-designed characters. Our approach integrates the exploration of a font-style latent space with human-in-the-loop preferential Bayesian optimization and multimodal references, facilitating efficient exploration and enhancing user control. Moreover, FontCraft allows users to revisit previous designs, retracting their earlier choices in the preferential Bayesian optimization process. Once users finish editing the style of a selected character, they can propagate it to the remaining characters and further refine them as needed. The system then generates a complete outline font in OpenType format. We evaluated the effectiveness of FontCraft through a user study comparing it to a baseline interface. Results from both quantitative and qualitative evaluations demonstrate that FontCraft enables non-expert users to design fonts efficiently.

Problem

• Automates font generation without pre-designed characters
• Enhances user control with Bayesian optimization
• Facilitates efficient font design for non-experts

Innovation

• Interactive Bayesian Optimization
• Multimodal references integration
• OpenType font generation