PointDGMamba: Domain Generalization of Point Cloud Classification via Generalized State Space Model

2024-08-24

arXiv.org

AI Summary

Point cloud classification suffers from poor cross-domain generalization, while conventional CNNs and Transformers face limitations in receptive field coverage or high computational complexity. Method: This paper proposes the first generalized State Space Model (SSM) framework tailored for point cloud domain generalization. It introduces three novel components: Masked Sequence Denoising, Sequence-wise Cross-domain Feature Aggregation, and Dual-level Domain Scanning. Additionally, we establish PointDG-3to1—a more challenging multi-source benchmark for point cloud domain generalization. Contribution/Results: The framework achieves global modeling with linear computational complexity, significantly improving generalization to unseen domains. It attains state-of-the-art performance across multiple cross-domain point cloud classification benchmarks, establishing a new paradigm for point cloud domain generalization.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Domain Generalization (DG) has been recently explored to improve the generalizability of point cloud classification (PCC) models toward unseen domains. However, they often suffer from limited receptive fields or quadratic complexity due to using convolution neural networks or vision Transformers. In this paper, we present the first work that studies the generalizability of state space models (SSMs) in DG PCC and find that directly applying SSMs into DG PCC will encounter several challenges: the inherent topology of the point cloud tends to be disrupted and leads to noise accumulation during the serialization stage. Besides, the lack of designs in domain-agnostic feature learning and data scanning will introduce unanticipated domain-specific information into the 3D sequence data. To this end, we propose a novel framework, PointDGMamba, that excels in strong generalizability toward unseen domains and has the advantages of global receptive fields and efficient linear complexity. PointDGMamba consists of three innovative components: Masked Sequence Denoising (MSD), Sequence-wise Cross-domain Feature Aggregation (SCFA), and Dual-level Domain Scanning (DDS). In particular, MSD selectively masks out the noised point tokens of the point cloud sequences, SCFA introduces cross-domain but same-class point cloud features to encourage the model to learn how to extract more generalized features. DDS includes intra-domain scanning and cross-domain scanning to facilitate information exchange between features. In addition, we propose a new and more challenging benchmark PointDG-3to1 for multi-domain generalization. Extensive experiments demonstrate the effectiveness and state-of-the-art performance of PointDGMamba.

Problem

• Point Cloud Classification
• Domain Generalization
• Global Information

Innovation

• PointDGMamba
• Noise Reduction
• Cross-Dataset Generalization