OmniMech: All-in-one Multimodal Mechanical Benchmark for 3D Reconstruction

2026-08-05

AI Summary

Existing vision-language models struggle to generate executable CAD programs that meet millimeter-level tolerance requirements. This work proposes OmniMech—the first million-scale multimodal benchmark for industrial mechanical design—integrating 2D engineering drawings, multi-view renders, meshes, STEP files, B-reps, and semantic annotations. It introduces a comprehensive task suite encompassing parametric modeling, geometric reasoning, annotation comprehension, and tool-augmented agent reasoning. For the first time, the benchmark enables fine-grained alignment of industrial data with complete dimensional and tolerance annotations, supported by an integrated toolchain for visualization, measurement, and CAD execution validation. Experiments reveal that current models still face significant challenges in synthesizing executable programs, achieving high-fidelity 3D reconstruction, and satisfying stringent tolerance constraints. The project will publicly release its dataset, evaluation code, and tool APIs to advance the application boundaries of vision-language models in high-precision manufacturing.

Abstract

Recent vision-language models (VLMs) can generate executable CAD programs from images, but existing methods mainly target coarse, general-purpose 3D objects and rarely address the fine-grained geometry and millimeter-level tolerances required in industrial mechanical design. We introduce OmniMech, the first million-scale benchmark for evaluating VLMs on executable CAD generation from industrial manufacturing data. OmniMech contains more than 251,000 fully dimensioned and toleranced 2D orthographic drawings, paired with native CAD models, multi-view renderings, mesh, STEP and B-rep representations, and rich semantic annotations. The benchmark includes four tasks: (1) parametric CAD program synthesis from engineering drawings; (2) diagram-to-3D reasoning for geometrically and structurally consistent reconstruction; (3) annotation-grounded reasoning over dimensions, symbols, feature callouts, and manufacturing constraints; and (4) tool-augmented agentic reasoning using visualization, measurement, CAD execution, and verification tools. Experiments show that current VLMs and CAD-specialized models still struggle with executable program synthesis, fine-grained 3D reconstruction, and reliable enforcement of dimensions and tolerances. We will release the benchmark data, evaluation code, and tool interfaces to support future research.

Problem

• executable CAD generation
• fine-grained geometry
• millimeter-level tolerances
• industrial mechanical design
• 3D reconstruction

Innovation

• OmniMech
• executable CAD generation
• multimodal mechanical benchmark
• dimensional tolerance reasoning
• tool-augmented agentic reasoning