GeoCAD: Local Geometry-Controllable CAD Generation

2025-06-12

AI Summary

Existing CAD generation methods lack fine-grained semantic-geometric alignment, rendering them incapable of supporting text-guided, geometry-controllable local edits (e.g., “isosceles right triangle”). This work introduces the first CAD generation framework tailored for local geometric editing. We propose a complementary local annotation strategy—combining hand-crafted vertex rules with vision-language large model (VLLM)-generated descriptions—to construct a 221K-sample geometric instruction dataset. Crucially, we pioneer the integration of large language models (LLMs) into local structure completion, unifying vertex geometry parsing, VLLM-driven local description generation, masked modeling, and LLM-conditioned structural prediction. Our method significantly outperforms state-of-the-art approaches in generation quality, geometric validity, and text-CAD alignment. It enables arbitrary-location editing and supports diverse predefined geometric constraints, substantially advancing automation and interactivity in industrial design.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models
• Natural Language Processing: (Large) Language Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Local geometry-controllable computer-aided design (CAD) generation aims to modify local parts of CAD models automatically, enhancing design efficiency. It also ensures that the shapes of newly generated local parts follow user-specific geometric instructions (e.g., an isosceles right triangle or a rectangle with one corner cut off). However, existing methods encounter challenges in achieving this goal. Specifically, they either lack the ability to follow textual instructions or are unable to focus on the local parts. To address this limitation, we introduce GeoCAD, a user-friendly and local geometry-controllable CAD generation method. Specifically, we first propose a complementary captioning strategy to generate geometric instructions for local parts. This strategy involves vertex-based and VLLM-based captioning for systematically annotating simple and complex parts, respectively. In this way, we caption $sim$221k different local parts in total. In the training stage, given a CAD model, we randomly mask a local part. Then, using its geometric instruction and the remaining parts as input, we prompt large language models (LLMs) to predict the masked part. During inference, users can specify any local part for modification while adhering to a variety of predefined geometric instructions. Extensive experiments demonstrate the effectiveness of GeoCAD in generation quality, validity and text-to-CAD consistency. Code will be available at https://github.com/Zhanwei-Z/GeoCAD.

Problem

• Modify local parts of CAD models automatically
• Follow user-specific geometric instructions accurately
• Overcome limitations in textual and local part focus

Innovation

• Local geometry-controllable CAD generation
• Complementary captioning for geometric instructions
• LLM-based prediction of masked CAD parts