PointDico: Contrastive 3D Representation Learning Guided by Diffusion Models

2025-12-09

AI Summary

The disorder and non-uniform density of 3D point clouds hinder self-supervised representation learning—contrastive models suffer from overfitting, while masked autoencoders exhibit limited modeling capacity. To address this, we propose DiffCL, the first framework that leverages diffusion models as guidance signals for contrastive learning. By integrating generative (diffusion) and discriminative (contrastive) learning via knowledge distillation, DiffCL introduces a hierarchical pyramid conditional generator and a dual-branch architecture to jointly model multi-scale geometric features and local-global contextual relationships. Cross-modal supervision is further incorporated to enhance generalization. Extensive experiments demonstrate state-of-the-art performance: 94.32% classification accuracy on ScanObjectNN and 86.5% instance mean IoU on ShapeNetPart—setting new benchmarks for 3D self-supervised representation learning.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Self-supervised representation learning has shown significant improvement in Natural Language Processing and 2D Computer Vision. However, existing methods face difficulties in representing 3D data because of its unordered and uneven density. Through an in-depth analysis of mainstream contrastive and generative approaches, we find that contrastive models tend to suffer from overfitting, while 3D Mask Autoencoders struggle to handle unordered point clouds. This motivates us to learn 3D representations by sharing the merits of diffusion and contrast models, which is non-trivial due to the pattern difference between the two paradigms. In this paper, we propose extit{PointDico}, a novel model that seamlessly integrates these methods. extit{PointDico} learns from both denoising generative modeling and cross-modal contrastive learning through knowledge distillation, where the diffusion model serves as a guide for the contrastive model. We introduce a hierarchical pyramid conditional generator for multi-scale geometric feature extraction and employ a dual-channel design to effectively integrate local and global contextual information. extit{PointDico} achieves a new state-of-the-art in 3D representation learning, extit{e.g.}, extbf{94.32%} accuracy on ScanObjectNN, extbf{86.5%} Inst. mIoU on ShapeNetPart.

Problem

• Develops a 3D representation learning model for unordered point clouds
• Integrates diffusion and contrastive models to overcome overfitting and density issues
• Achieves state-of-the-art accuracy on 3D object classification and segmentation

Innovation

• Integrates diffusion and contrastive models via knowledge distillation
• Uses hierarchical pyramid generator for multi-scale feature extraction
• Employs dual-channel design to combine local and global context