Multi-View In-Cabin Monitoring System for Public Transport Vehicles

2026-06-10

AI Summary

This work addresses the lack of synchronized, well-annotated multi-view RGB-D and LiDAR datasets in public transit cabins, which has hindered research on occupant state understanding and 3D perception. To bridge this gap, we present the first multimodal, synchronized dataset specifically designed for bus interiors, integrating four RGB-D cameras and a rotating LiDAR sensor. The dataset comprises 9,136 meticulously annotated samples and is accompanied by a complete pipeline including sensor calibration, end-to-end pseudo-label generation, and conversion to the nuScenes format. It enables training and evaluation of state-of-the-art multi-view 3D detection models such as Lift-Splat-Shoot and BEVFusion, significantly advancing research and applications in high-precision 3D human pose estimation and oriented bounding box prediction within cabin environments.

Abstract

We introduce a multi-view in-cabin monitoring dataset for public transportation with synchronized RGB and depth images from four inward-facing cameras and a rotating LiDAR covering the vehicle interior of a digitalized and partly automated German city bus. The dataset contains 9.136 synchronized samples with annotations and is accompanied by a calibration and pseudo-labeling pipeline that generates 3D human pose estimates and oriented 3D bounding boxes for occupants. We further provide a nuScenes-format conversion and benchmark representative multi-view 3D detection models (e.g., Lift-Splat-Shoot and BEVFusion), supporting comparative evaluation and small-scale training of multi-view in-cabin perception models. The dataset and tools are available at https://github.com/EvgenyGorelik/multiview_incabin_dataset.

Problem

• multi-view
• in-cabin monitoring
• 3D detection
• public transport
• dataset

Innovation

• multi-view perception
• in-cabin monitoring
• 3D human pose estimation
• oriented 3D bounding boxes
• LiDAR-RGB fusion