ERUnderstand: Evaluating Vision-Language Models on Structured ER Diagrams

2026-07-27

AI Summary

This work addresses the limitation that existing entity-relationship diagrams (ERDs) are predominantly image-based and lack machine-readable structured representations, hindering the application of AI in database engineering. To bridge this gap, we introduce ERUnderstand, the first large-scale multimodal benchmark specifically designed for structured understanding of ER diagrams, comprising 2,960 diverse diagrams sourced from educational, real-world, and synthetic scenarios to enable fine-grained evaluation. We establish standardized data representations, an evaluation toolkit, and a generation pipeline, and conduct systematic assessments of vision-language models (VLMs). Experiments show that leading VLMs achieve F1 scores above 0.74 on common ER elements but exhibit significant performance drops on weak entities (0.28), multi-valued attributes (0.14), and n-ary relationships (0.07). While reasoning-augmented strategies yield overall improvements of 15–25%, model performance remains constrained by linguistic priors and diagram complexity.

Abstract

Entity-Relationship Diagrams (ERDs) are central to conceptual database design, yet they are typically available only as rendered images rather than machine-readable schemas, limiting AI-assisted database engineering. We introduce ERUnderstand, the first large-scale benchmark for structured understanding of ER diagrams, comprising 2,960 diagrams collected from curated educational sources, real-world schemas, and synthetically generated examples spanning diverse domains, notations, complexity levels, and Extended Entity-Relationship (EER) constructs. Each diagram is paired with a standardized machine-readable representation for fine-grained evaluation of schema elements. Evaluating state-of-the-art Vision-Language Models (VLMs), we find that while common ERD elements are recovered reliably (F1 > 0.74), performance drops sharply on weak entities (as low as 0.28 F1), multivalued attributes (0.14 F1), and N-ary relationships (0.07 F1). Reasoning-augmented models improve overall performance by 15-25% but remain sensitive to linguistic priors and increasing diagram complexity. ERUnderstand provides a standardized benchmark for evaluating multimodal understanding of conceptual database schemas. The benchmark, dataset, evaluation toolkit, and generation code are publicly available at https://github.com/salinaria/ERUnderstand.

Problem

• Entity-Relationship Diagrams
• Vision-Language Models
• structured understanding
• machine-readable schemas
• database engineering

Innovation

• Entity-Relationship Diagrams
• Vision-Language Models
• Structured Understanding
• Multimodal Benchmark
• Database Schema