Automated Gaze-based Behavioral Segmentation and Temporal Representation for Bridge Inspection in Unconstrained 3D Environments

2026-06-12

AI Summary

This study addresses the subjectivity inherent in traditional visual bridge inspections and the lack of objective quantification of inspectors’ behavioral processes, particularly within complex, unconstrained 3D environments. By integrating multimodal data—including eye movements, head motion, drone navigation trajectories, and scene geometry—the work proposes a novel framework that segments temporal windows and applies machine learning to automatically classify inspection behaviors into three distinct modes: global scanning, local examination, and navigation. This approach significantly enhances ecological validity and enables the extraction of interpretable behavioral metrics, such as fixation duration, transition probabilities, and spatial revisitation rates. Validated on a virtual bridge inspection platform, the method effectively differentiates between inspection strategies and provides preliminary evidence linking specific behavioral patterns to inspection performance.

Abstract

Visual bridge inspection is a knowledge-intensive task in which inspectors coordinate visual search, spatial navigation, structural reasoning, and defect identification and documentation. It is a central maintenance task for bridges and a key basis for safety assessments, yet its results are susceptible to individual subjectivity. While eye-tracking-based behavioral studies quantify underlying processes, existing research often imposes restrictive simplifications to reduce environmental complexity, thereby compromising ecological validity. This study proposes an automated data analytics framework for converting multimodal inspection data into an inspection mode time series. Unconstrained 3D gaze, head-movement, drone navigation, and scene geometry data are segmented into temporal windows and classified into three functional modes: global scanning, local inspection, and navigation. The resulting temporal representation enables the extraction of interpretable behavioral descriptors, including transition probabilities, dwell times, transition entropy, fixation measures, and spatial revisit metrics. A feasibility study using a virtual bridge inspection platform demonstrates that the proposed representation captures meaningful differences in inspection strategy and reveals exploratory relationships with inspection performance. This study contributes a framework for human-informed computer-aided infrastructure inspection systems, inspector training, and data-driven assessment of constructed facilities.

Problem

• bridge inspection
• gaze-based behavior
• ecological validity
• 3D environments
• subjectivity

Innovation

• gaze-based behavioral segmentation
• temporal representation
• unconstrained 3D environments
• multimodal inspection data
• inspection mode classification