Generative Point Cloud Registration

2025-12-10

AI Summary

To address the challenges of cross-view geometric-color feature alignment and poor matching robustness in 3D point cloud registration, this paper proposes a generative paradigm: leveraging controllable 2D generative models to synthesize cross-view-consistent image pairs for depth-guided joint geometric-color optimization. Key contributions include: (1) the first integration of 2D generative models into a 3D registration framework; (2) Match-ControlNet—a novel architecture that unifies texture-consistent image generation and geometric alignment via depth-map projection, coupled denoising, and prompt-guided conditioning; and (3) a lightweight geometric-color feature fusion module. Extensive experiments on 3DMatch and ScanNet demonstrate significant improvements in registration accuracy and noise robustness. Moreover, our method is plug-and-play, effectively enhancing the performance of mainstream registration approaches without architectural modifications.

Technology Category

Computer Vision: 3D Computer VisionIntelligent Robots: Multimodal Perception & Sensor FusionMachine Learning: Deep Generative Models & Autoencoders

Application Category

Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingGraph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

In this paper, we propose a novel 3D registration paradigm, Generative Point Cloud Registration, which bridges advanced 2D generative models with 3D matching tasks to enhance registration performance. Our key idea is to generate cross-view consistent image pairs that are well-aligned with the source and target point clouds, enabling geometry-color feature fusion to facilitate robust matching. To ensure high-quality matching, the generated image pair should feature both 2D-3D geometric consistency and cross-view texture consistency. To achieve this, we introduce Match-ControlNet, a matching-specific, controllable 2D generative model. Specifically, it leverages the depth-conditioned generation capability of ControlNet to produce images that are geometrically aligned with depth maps derived from point clouds, ensuring 2D-3D geometric consistency. Additionally, by incorporating a coupled conditional denoising scheme and coupled prompt guidance, Match-ControlNet further promotes cross-view feature interaction, guiding texture consistency generation. Our generative 3D registration paradigm is general and could be seamlessly integrated into various registration methods to enhance their performance. Extensive experiments on 3DMatch and ScanNet datasets verify the effectiveness of our approach.

Problem

• Bridges 2D generative models with 3D point cloud registration tasks.
• Generates cross-view consistent image pairs for geometry-color feature fusion.
• Ensures 2D-3D geometric and cross-view texture consistency in matching.

Innovation

• Generative 3D registration bridges 2D generative models with 3D matching
• Match-ControlNet ensures geometric and texture consistency for robust matching
• General paradigm integrates into various methods to enhance performance