🤖 AI Summary
This work addresses the limitation of existing CAD model evaluation methods, which predominantly emphasize visual fidelity while neglecting engineering functionality. To bridge this gap, the authors propose CADEngBench, a dual-track benchmark that systematically incorporates engineering behavior validation—including finite element analysis (FEA) alignment, design-for-manufacturing (DFM) checks, and kinematic joint dynamics—into the assessment framework, covering both parametric parts and assemblies. The benchmark employs techniques such as B-Rep validity verification, parameter perturbation tests, functional editing tasks, and linear static simulations using CalculiX to comprehensively evaluate the engineering-grade capabilities of generated and edited models. Experimental results reveal that while current multimodal models outperform in CAD editing over generation, they still struggle with complex edits, FEA consistency, and accurately reconstructing real-world assembly mating relationships.
📝 Abstract
A CAD model is not engineering-grade merely because it looks correct. It must satisfy design requirements, respond predictably to parameter changes, support controlled edits, match a reference structural response under a declared analysis, and connect to other parts through valid joints. We present CADEngBench, a two-track benchmark for these capabilities. CADEngBench-P evaluates 300 parametric parts, each used for one zero-to-CAD task and one functional-editing task (600 tasks in total), through boundary-representation (B-Rep) validity, engineering and DFM checks, parameter-family perturbations, functional editing, and matched linear-static FEA in CalculiX. CADEngBench-A evaluates 150 body pairs through ranked joint retrieval, exact face-and-edge grounding, joint-frame prediction, and kinematic verification. Across eight multimodal, code-capable models, editing supplied CAD is substantially easier than generating it, while complex edits and matched FEA remain difficult. Assembly predictions often locate the relevant region but fail to recover the recorded joint or mating entities. These results show that CAD evaluation must test engineering behavior rather than appearance alone.