CADmium: Fine-Tuning Code Language Models for Text-Driven Sequential CAD Design

2025-07-13

AI Summary

CAD modeling remains heavily reliant on manual intervention, resulting in low efficiency; existing approaches lack text-driven, sequential automated design capabilities. Method: We introduce the first large-scale, high-quality dataset for text-to-CAD generation—comprising over 170,000 samples—where GPT-4.1 generates high-fidelity natural-language descriptions and CAD operation sequences are structured in JSON format. We fine-tune code-large language models via instruction tuning, leveraging both synthetic and human-annotated data, with ablation studies validating design choices. Contribution/Results: We propose novel structural quality metrics—sphericity, mean curvature, and Euler characteristic—that capture geometric and topological fidelity, addressing limitations of conventional metrics. Experiments demonstrate substantial improvements in accuracy and practical utility of text-to-CAD generation, enabling rapid modeling across diverse categories. This work establishes the feasibility of LLMs for industrial design automation. The model, dataset, and code are fully open-sourced.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Computer Vision: Diffusion Models for Vision

Application Category

• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Computer-aided design (CAD) is the digital construction of 2D and 3D objects, and is central to a wide range of engineering and manufacturing applications like automobile and aviation. Despite its importance, CAD modeling remains largely a time-intensive, manual task. Recent works have attempted to automate this process with small transformer-based models and handcrafted CAD sequence representations. However, there has been little effort to leverage the potential of large language models (LLMs) for sequential CAD design. In this work, we introduce a new large-scale dataset of more than 170k CAD models annotated with high-quality, human-like descriptions generated with our pipeline based on GPT-4.1. Using this dataset, we fine-tune powerful code-LLMs to generate CAD sequences represented in a JSON-based format from natural language descriptions, demonstrating the viability and effectiveness of this approach for text-conditioned CAD generation. Because simple metrics often fail to reflect the quality of generated objects, we introduce geometric and topological metrics based on sphericity, mean curvature, and Euler characteristic to provide richer structural insights. Our experiments and ablation studies on both synthetic and human-annotated data demonstrate that CADmium is able to automate CAD design, drastically speeding up the design of new objects. The dataset, code, and fine-tuned models are available online.

Problem

• Automating time-intensive manual CAD modeling tasks
• Leveraging LLMs for sequential CAD design from text
• Evaluating CAD generation quality with advanced geometric metrics

Innovation

• Fine-tune code-LLMs for CAD sequence generation
• Introduce JSON-based CAD representation from text
• Propose geometric metrics for quality evaluation