Wukong's 72 Transformations: High-fidelity Textured 3D Morphing via Flow Models

2025-11-27

AI Summary

This work addresses the challenge of high-fidelity, training-free 3D shape-and-texture morphing—bypassing conventional reliance on manual correspondence establishment and explicit deformation trajectory estimation. The proposed method formulates morphing as an optimal transport–based barycentric interpolation problem. It integrates a streaming Transformer to generate geometric priors, similarity-guided semantic consistency constraints, and a progressive sequence initialization strategy. Given only source/target text or image prompts, it achieves joint, smooth geometric and textural transitions without registration or supervision, effectively suppressing oversmoothing artifacts. Qualitative and quantitative evaluations across diverse shape-texture co-morphing tasks demonstrate superior performance over existing unsupervised and weakly supervised approaches. To our knowledge, this is the first method enabling purely prompt-driven, end-to-end consistent, high-quality 3D semantic morphing.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

We present WUKONG, a novel training-free framework for high-fidelity textured 3D morphing that takes a pair of source and target prompts (image or text) as input. Unlike conventional methods -- which rely on manual correspondence matching and deformation trajectory estimation (limiting generalization and requiring costly preprocessing) -- WUKONG leverages the generative prior of flow-based transformers to produce high-fidelity 3D transitions with rich texture details. To ensure smooth shape transitions, we exploit the inherent continuity of flow-based generative processes and formulate morphing as an optimal transport barycenter problem. We further introduce a sequential initialization strategy to prevent abrupt geometric distortions and preserve identity coherence. For faithful texture preservation, we propose a similarity-guided semantic consistency mechanism that selectively retains high-frequency details and enables precise control over blending dynamics. This avoids common artifacts like oversmoothing while maintaining semantic fidelity. Extensive quantitative and qualitative evaluations demonstrate that WUKONG significantly outperforms state-of-the-art methods, achieving superior results across diverse geometry and texture variations.

Problem

• Generates high-fidelity textured 3D morphing from prompts without training
• Ensures smooth shape transitions by formulating morphing as optimal transport
• Preserves texture details and semantic fidelity to avoid artifacts

Innovation

• Training-free framework using flow-based transformers
• Optimal transport barycenter for smooth shape transitions
• Similarity-guided semantic consistency for texture preservation