DG-MVP: 3D Domain Generalization via Multiple Views of Point Clouds for Classification

2025-04-16

AI Summary

This work addresses the severe domain shift between CAD-synthetic and LiDAR-real 3D point clouds in cross-domain classification, primarily caused by occlusion and incompleteness. To mitigate this, we propose a domain generalization method based on multi-view 2D projection representations. Unlike conventional point-based feature extractors (e.g., PointNet variants), which suffer from local feature loss due to max-pooling, our approach renders point clouds into multi-channel 2D images from multiple viewpoints and leverages 2D convolutional networks to efficiently capture domain-invariant geometric structures. Integrated with a domain generalization training paradigm, the method achieves substantial improvements on the PointDA-10 and Sim-to-Real benchmarks—yielding up to a 5.2% absolute gain in cross-domain classification accuracy. To our knowledge, this is the first work to empirically validate the effectiveness and robustness of 2D projection representations for point cloud domain generalization.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: 3D Computer Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Deep neural networks have achieved significant success in 3D point cloud classification while relying on large-scale, annotated point cloud datasets, which are labor-intensive to build. Compared to capturing data with LiDAR sensors and then performing annotation, it is relatively easier to sample point clouds from CAD models. Yet, data sampled from CAD models is regular, and does not suffer from occlusion and missing points, which are very common for LiDAR data, creating a large domain shift. Therefore, it is critical to develop methods that can generalize well across different point cloud domains. %In this paper, we focus on the 3D point cloud domain generalization problem. Existing 3D domain generalization methods employ point-based backbones to extract point cloud features. Yet, by analyzing point utilization of point-based methods and observing the geometry of point clouds from different domains, we have found that a large number of point features are discarded by point-based methods through the max-pooling operation. This is a significant waste especially considering the fact that domain generalization is more challenging than supervised learning, and point clouds are already affected by missing points and occlusion to begin with. To address these issues, we propose a novel method for 3D point cloud domain generalization, which can generalize to unseen domains of point clouds. Our proposed method employs multiple 2D projections of a 3D point cloud to alleviate the issue of missing points and involves a simple yet effective convolution-based model to extract features. The experiments, performed on the PointDA-10 and Sim-to-Real benchmarks, demonstrate the effectiveness of our proposed method, which outperforms different baselines, and can transfer well from synthetic domain to real-world domain.

Problem

• Address domain shift in 3D point cloud classification
• Reduce reliance on annotated LiDAR data
• Improve generalization across synthetic and real-world domains

Innovation

• Uses multiple 2D projections of 3D point clouds
• Employs convolution-based model for feature extraction
• Addresses domain shift from synthetic to real data