3DSES: an indoor Lidar point cloud segmentation dataset with real and pseudo-labels from a 3D model

2025-01-29

AI Summary

Low semantic segmentation accuracy and high annotation cost for BIM-critical indoor components (e.g., lighting fixtures, security devices) in terrestrial laser scanning (TLS) point clouds. Method: We introduce the first BIM-oriented indoor point cloud semantic segmentation dataset (427 m²), featuring both point-level ground truth and complete 3D CAD model annotations. We propose a CAD model-driven automatic alignment and pseudo-label generation method achieving >95% labeling accuracy. We design three complexity-variant dataset splits and systematically evaluate— for the first time—the contribution of LiDAR intensity to BIM facility segmentation. Leveraging rigid registration, geometric projection, and intensity-coordinate fusion within PointNet++/KPConv frameworks, we significantly improve segmentation performance for BIM-relevant classes. Contribution/Results: All data, source code, and CAD models are publicly released to advance research in indoor semantic understanding and digital twin construction.

Technology Category

• Computer Vision: Segmentation
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Semantics and Knowledge: Representation, semantic annotation, enhancement, enrichments, access and/or integration of a variety of data on the Web
• Security and Privacy: Data transparency and provenance
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Semantic segmentation of indoor point clouds has found various applications in the creation of digital twins for robotics, navigation and building information modeling (BIM). However, most existing datasets of labeled indoor point clouds have been acquired by photogrammetry. In contrast, Terrestrial Laser Scanning (TLS) can acquire dense sub-centimeter point clouds and has become the standard for surveyors. We present 3DSES (3D Segmentation of ESGT point clouds), a new dataset of indoor dense TLS colorized point clouds covering 427 m 2 of an engineering school. 3DSES has a unique double annotation format: semantic labels annotated at the point level alongside a full 3D CAD model of the building. We introduce a model-to-cloud algorithm for automated labeling of indoor point clouds using an existing 3D CAD model. 3DSES has 3 variants of various semantic and geometrical complexities. We show that our model-to-cloud alignment can produce pseudo-labels on our point clouds with a > 95% accuracy, allowing us to train deep models with significant time savings compared to manual labeling. First baselines on 3DSES show the difficulties encountered by existing models when segmenting objects relevant to BIM, such as light and safety utilities. We show that segmentation accuracy can be improved by leveraging pseudo-labels and Lidar intensity, an information rarely considered in current datasets. Code and data will be open sourced.

Problem

• Point Cloud Processing
• Indoor Environment
• Object Recognition

Innovation

• 3DSES Dataset
• Auto-Labeling Algorithm
• Lidar Technology