SafeBuild-Bench: A Temporal-Robust Construction Safety Benchmark with Graph-Enhanced Data Mining

2026-07-29

AI Summary

This study addresses the limitations of existing construction safety models, which struggle to handle deployment risks arising from temporal evolution and site dynamics in real-world scenarios and lack robust evaluation benchmarks for redundant and long-tailed industrial imagery. To bridge this gap, we propose the Graph-Enhanced Multimodal Selection pipeline (GEMS), which uniquely integrates proxy model confusion signals with graph-based diversity strategies to efficiently mine high-quality samples from over 100,000 industrial image-text records and automate expert validation. Leveraging this approach, we construct SafeBuild-Bench—the first multimodal benchmark for construction safety that explicitly accounts for temporal and spatial variations—comprising 3,314 instances spanning multiple-choice and free-form description tasks. Experiments reveal that state-of-the-art multimodal large language models achieve only around 60% accuracy on this benchmark, highlighting their substantial shortcomings in practical safety comprehension.

Abstract

Construction-safety models must handle concrete deployment risks, such as a worker standing near a scaffold edge without guardrails, rather than only recognize common objects in curated images. Yet real inspection archives are redundant, long-tailed, and collected across changing sites and months. We introduce SafeBuild-Bench, a metadata-driven benchmark for evaluating multimodal large language models on construction safety under realistic temporal and site variation. It is mined from 100K+ industrial image-text records and contains 3,314 task instances from over 3,000 expert-verified images, covering multiple-choice hazard identification and free-form hazard description. To make expert verification scalable, we develop GEMS, a graph-enhanced multimodal selection pipeline that combines a proxy-model confusion signal with graph-based diversity to identify informative candidates from redundant streams. On public instruction-tuning data, GEMS-selected subsets preserve robustness-oriented performance under small data budgets. On SafeBuild-Bench, current MLLMs remain far from reliable construction-safety understanding, with the best overall score near 60. We release the benchmark, evaluation scripts, and GEMS codebase at https://github.com/safebuild/gems.

Problem

• construction safety
• temporal robustness
• multimodal large language models
• hazard identification
• real-world benchmark

Innovation

• construction safety
• temporal robustness
• graph-enhanced data mining
• multimodal large language models
• expert-verified benchmark