Occlusion-aware Driver Monitoring System using the Driver Monitoring Dataset

2025-04-29

AI Summary

Driver Monitoring Systems (DMS) suffer from weak situational awareness and poor occlusion robustness under multi-illumination conditions—especially low-light scenarios. Method: This paper proposes an occlusion-aware DMS framework fusing RGB and infrared (IR) modalities. It introduces, for the first time in DMS, a lightweight real-time facial occlusion detection module; constructs independently trained RGB and IR dual-stream models; integrates multi-region gaze estimation with occlusion classification; and designs a cross-sensor algorithmic fusion pipeline. Contribution/Results: The framework complies with Euro NCAP requirements and achieves a 12.3% absolute improvement in occlusion detection accuracy over unimodal baselines. Extensive evaluation on both real-vehicle and DMD datasets demonstrates reliable joint performance across identity recognition, regional gaze estimation, and occlusion detection under challenging illumination conditions—thereby significantly enhancing overall situational awareness and system trustworthiness.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Application Domains: Mobility, Driving & Flight

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Responsible Web: Algorithmic accountability and transparency on the web

Abstract

This paper presents a robust, occlusion-aware driver monitoring system (DMS) utilizing the Driver Monitoring Dataset (DMD). The system performs driver identification, gaze estimation by regions, and face occlusion detection under varying lighting conditions, including challenging low-light scenarios. Aligned with EuroNCAP recommendations, the inclusion of occlusion detection enhances situational awareness and system trustworthiness by indicating when the system's performance may be degraded. The system employs separate algorithms trained on RGB and infrared (IR) images to ensure reliable functioning. We detail the development and integration of these algorithms into a cohesive pipeline, addressing the challenges of working with different sensors and real-car implementation. Evaluation on the DMD and in real-world scenarios demonstrates the effectiveness of the proposed system, highlighting the superior performance of RGB-based models and the pioneering contribution of robust occlusion detection in DMS.

Problem

• Develops occlusion-aware driver monitoring for varying conditions
• Integrates RGB and IR algorithms for reliable performance
• Enhances system trustworthiness with robust occlusion detection

Innovation

• Occlusion-aware DMS with RGB and IR algorithms
• Driver identification and gaze estimation by regions
• Robust face occlusion detection in varying lighting