Don't Mesh with Me: Generating Constructive Solid Geometry Instead of Meshes by Fine-Tuning a Code-Generation LLM

2024-11-22

arXiv.org

AI Summary

Existing mesh-based generative methods for mechanical engineering lack the precision and parametric editability required for high-fidelity CAD design. Method: We propose the first end-to-end, natural language–driven framework for generating constructive solid geometry (CSG) code tailored to CAD workflows—bypassing low-fidelity mesh representations to directly produce syntactically correct, geometrically valid, and CAD-importable parametric CSG scripts in Python. Contribution/Results: Our approach introduces three key innovations: (1) the first application of large language models (LLMs) to CSG code generation; (2) the construction of the first publicly available Python dataset mapping boundary representation (BREP) geometries to CSG programs, annotated with natural language descriptions (validated by GPT-4 and refined manually); and (3) a pipeline integrating BREP geometric parsing, CSG compilation, and supervised fine-tuning (SFT) to map semantic descriptions and spatial constraints to executable geometric code. Experiments demonstrate substantial improvements in both automation capability and modeling accuracy for mechanical design.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

While recent advancements in machine learning, such as LLMs, are revolutionizing software development and creative industries, they have had minimal impact on engineers designing mechanical parts, which remains largely a manual process. Existing approaches to generating 3D geometry most commonly use meshes as a 3D representation. While meshes are suitable for assets in video games or animations, they lack sufficient precision and adaptability for mechanical engineering purposes. This paper introduces a novel approach for the generation of 3D geometry that generates surface-based Constructive Solid Geometry (CSG) by leveraging a code-generation LLM. First, we create a dataset of 3D mechanical parts represented as code scripts by converting Boundary Representation geometry (BREP) into CSG-based Python scripts. Second, we create annotations in natural language using GPT-4. The resulting dataset is used to fine-tune a code-generation LLM. The fine-tuned LLM can complete geometries based on positional input and natural language in a plausible way, demonstrating geometric understanding.

Problem

• Generating precise 3D geometry for mechanical engineering using CSG instead of meshes
• Converting BREP to CSG-based Python scripts for dataset creation
• Fine-tuning LLM to complete geometries from positional and natural language input

Innovation

• Fine-tune code-generation LLM for CSG
• Convert BREP to CSG-based Python scripts
• Generate CSG from natural language input