CANIS: Generation-Assisted 3D Canonicalization via an Image-Semantic Bridge

2026-08-07

AI Summary

This work addresses the challenge of achieving semantically consistent 3D object orientation normalization, a task where existing methods relying primarily on geometric cues often fall short. The paper introduces the first general-purpose framework that requires neither category-specific templates nor dedicated training, instead leveraging the semantic priors embedded in a frozen image-to-3D generative model. By rendering candidate viewpoints, constructing structure-preserving proxy models, and using rendered images as semantic bridges to establish correspondences between input shapes and proxies, the method jointly estimates a canonical rigid transformation through depth back-projection and geometric constraints. Validated on synthetic data and demonstrating strong generalization under partial observations and real-world scans, the approach significantly improves performance across downstream tasks—including 3D classification, part segmentation, and dense correspondence—under arbitrary object rotations.

Abstract

Canonicalizing 3D object orientation is fundamental to 3D understanding and analysis. Existing approaches often rely on geometric cues, although 3D canonicalization ultimately requires a semantically meaningful orientation. To address this gap, we propose CANIS, a category-agnostic, generation-assisted framework that introduces the semantic orientation prior of a frozen image-to-3D generative model into 3D canonicalization, without canonicalization-specific training or category-specific templates. Specifically, CANIS first renders the input object from candidate viewpoints, selects an informative view, and generates a proxy in a canonical orientation. During generation, a sparse structural latent encoded from the input guides the proxy to preserve the geometry of an object. CANIS then uses the selected image as a semantic bridge between the input and the proxy. Image patches identify semantic regions on the proxy, and depth back-projection locates the corresponding regions on the input. The resulting semantic anchors constrain geometric matching, from which we estimate the rigid transformation that canonicalizes the input. Experiments on synthetic benchmarks validate CANIS and its key components, while qualitative results on partial observations and OmniObject3D suggest its applicability to incomplete and real-world scans. CANIS also improves downstream 3D classification, part segmentation, and dense correspondence under arbitrary rotations. Project page: https://kenkenzaii.github.io/Canis.

Problem

• 3D canonicalization
• semantic orientation
• geometric cues
• category-agnostic
• 3D understanding

Innovation

• 3D canonicalization
• generation-assisted
• semantic orientation prior
• category-agnostic
• image-semantic bridge