CAGE-GS: High-fidelity Cage Based 3D Gaussian Splatting Deformation

2025-04-17

AI Summary

To address geometric distortion and texture blurring in 3D Gaussian Splatting (3DGS) deformation, this paper proposes a high-fidelity deformation method based on a learned control cage. The method tackles the problem of aligning source 3DGS representations with diverse multimodal targets—including text, images, point clouds, meshes, or other 3DGS instances—while preserving structural integrity and visual fidelity. Its core contributions are: (1) an end-to-end learnable deformable control cage that enables precise, topology-aware alignment across modalities; and (2) a Jacobian-driven covariance adaptation mechanism that ensures geometric consistency and maintains Gaussian ellipsoid texture fidelity under deformation. Evaluated on both public and custom-built benchmarks, the approach achieves state-of-the-art performance in deformation accuracy, detail preservation, and computational efficiency, enabling real-time interactive editing.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

As 3D Gaussian Splatting (3DGS) gains popularity as a 3D representation of real scenes, enabling user-friendly deformation to create novel scenes while preserving fine details from the original 3DGS has attracted significant research attention. We introduce CAGE-GS, a cage-based 3DGS deformation method that seamlessly aligns a source 3DGS scene with a user-defined target shape. Our approach learns a deformation cage from the target, which guides the geometric transformation of the source scene. While the cages effectively control structural alignment, preserving the textural appearance of 3DGS remains challenging due to the complexity of covariance parameters. To address this, we employ a Jacobian matrix-based strategy to update the covariance parameters of each Gaussian, ensuring texture fidelity post-deformation. Our method is highly flexible, accommodating various target shape representations, including texts, images, point clouds, meshes and 3DGS models. Extensive experiments and ablation studies on both public datasets and newly proposed scenes demonstrate that our method significantly outperforms existing techniques in both efficiency and deformation quality.

Problem

• Enables deformation of 3D Gaussian Splatting scenes
• Preserves texture fidelity during geometric transformation
• Supports diverse target shape representations

Innovation

• Cage-based 3DGS deformation method
• Jacobian matrix updates covariance parameters
• Flexible target shape representation support