TopoGaussian: Inferring Internal Topology Structures from Visual Clues

2025-03-16

AI Summary

This work addresses the challenge of reconstructing internal topological structures of opaque objects. We propose an end-to-end differentiable framework that bypasses conventional mesh-based volume filling and repair. Our core innovation is a novel mesh-free, multi-physics particle simulator—integrating constitutive modeling, actuation dynamics, and collision response—coupled with Gaussian rasterization, neural implicit surfaces, and quadric-based geometric representations to enable unified gradient optimization across particle, implicit, and analytic geometry domains. The method directly infers internal structure from standard photographs and videos. Evaluated on synthetic data and four real-world 3D-printing tasks, it achieves an average 5.26× speedup over traditional mesh-based approaches while significantly improving reconstruction accuracy and geometric fidelity. By eliminating mesh dependency, our framework overcomes fundamental limitations in modeling complex internal topologies and establishes a new paradigm for 3D vision, soft robotics, and intelligent manufacturing.

Technology Category

• Computer Vision: Low Level & Physics-based Vision
• Search and Optimization: Sampling/Simulation-based Search
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

We present TopoGaussian, a holistic, particle-based pipeline for inferring the interior structure of an opaque object from easily accessible photos and videos as input. Traditional mesh-based approaches require tedious and error-prone mesh filling and fixing process, while typically output rough boundary surface. Our pipeline combines Gaussian Splatting with a novel, versatile particle-based differentiable simulator that simultaneously accommodates constitutive model, actuator, and collision, without interference with mesh. Based on the gradients from this simulator, we provide flexible choice of topology representation for optimization, including particle, neural implicit surface, and quadratic surface. The resultant pipeline takes easily accessible photos and videos as input and outputs the topology that matches the physical characteristics of the input. We demonstrate the efficacy of our pipeline on a synthetic dataset and four real-world tasks with 3D-printed prototypes. Compared with existing mesh-based method, our pipeline is 5.26x faster on average with improved shape quality. These results highlight the potential of our pipeline in 3D vision, soft robotics, and manufacturing applications.

Problem

• Infer internal topology from photos and videos
• Overcome limitations of traditional mesh-based methods
• Optimize topology using flexible representation choices

Innovation

• Particle-based pipeline for topology inference
• Combines Gaussian Splatting with differentiable simulator
• Flexible topology representation for optimization