MechVQA: Benchmarking and Enhancing Multimodal LLMs on Comprehensive Mechanical Drawing Understanding

2026-05-28

AI Summary

This study addresses the challenges faced by multimodal large language models in understanding mechanical engineering drawings—namely, densely packed annotations, limited domain-specific knowledge, and stringent geometric constraints that complicate spatial reasoning. To tackle these issues, the authors introduce MechVQA, the first comprehensive dataset for mechanical drawing comprehension, and propose MechVL, a multimodal vision-language model trained in multiple stages that integrates domain knowledge from mechanical drafting with multimodal learning. The work also establishes a fine-grained evaluation benchmark spanning three capability tiers: recognition, reasoning, and judgment, supported by a semi-automated data construction and quality control pipeline. Experimental results demonstrate that MechVL outperforms the strongest closed-source baseline by 7.57 percentage points on MechVQA, substantially enhancing the practical applicability of such models in mechanical design and inspection scenarios.

Abstract

Multimodal Large Language Models (MLLMs) have demonstrated significant achievements in general visual question answering (VQA) tasks. However, they remain brittle on mechanical engineering drawings, where high annotation density and weak domain knowledge, compounded by unreliable spatial relation reasoning under strict projection rules and geometric constraints, make decisive cues easy to miss and frequently lead to wrong answers. To bridge this gap, we introduce the first comprehensive mechanical drawing understanding dataset, MechVQA, created through a semi-automated construction and quality-control pipeline. MechVQA contains 3.3k high-density pictures with 21K question-answer pairs, spanning 10 different fine-grained tasks across three capability levels: Recognition, Reasoning, and Judging, providing a testbed to evaluate and improve MLLM understanding on real-world mechanical drawings. On top of MechVQA, we then develop the MechVL model through a multi-stage training paradigm, building a strong domain-specialized baseline. Extensive experimental results demonstrate that MechVL outperforms the strongest closed-source baseline by 7.57 percentage points on the MechVQA total score, significantly enhancing mechanical drawing understanding ability and providing a reusable foundation for deploying MLLMs in mechanical design and inspection scenarios.

Problem

• Mechanical Drawing Understanding
• Multimodal Large Language Models
• Visual Question Answering
• Spatial Relation Reasoning
• Domain Knowledge

Innovation

• Mechanical Drawing Understanding
• Multimodal LLMs
• MechVQA Dataset
• Spatial Relation Reasoning
• Domain-Specialized Training