SPLATIFY: Reproduce, Discover, Innovate! From Papers and Ideas to Trainable 3DGS Code

2026-10-06

AI Summary

This work addresses the prolonged reproduction cycles of 3D Gaussian Splatting (3DGS) papers and the frequent failure to generate code for state-of-the-art models by proposing a multi-agent framework that automatically translates 3DGS papers into trainable code. Methodologically, the framework integrates context-free grammar constraints, graph-of-thought reasoning, retrieval-augmented generation (RAG), and vision-language model (VLM) visual feedback mechanisms to enable efficient code synthesis, autonomous error correction, and combinatorial innovation. Experimental results demonstrate that this approach reduces reproduction time from weeks to minutes while improving peak signal-to-noise ratio (PSNR) by up to 2.4 dB. Furthermore, the framework successfully facilitates cross-disciplinary knowledge transfer, autonomously generating novel methods for entirely new scientific domains such as nebula rendering.

Abstract

The rapid growth of 3D Gaussian Splatting (3DGS) research demands significant effort to reimplement papers before building on them. We introduce SPLATIFY, a multi-agent framework that converts 3DGS papers into trainable gsplat-based implementations, where generic paper-to-code methods and frontier models fail. SPLATIFY achieves this through five innovations: (1) A context-free grammar for gsplat over a modular method template with extension points for losses, densification, rendering, and optimization, constraining synthesis so generated code satisfies gsplat's architectural invariants by construction. (2) Architectural elements for faithful reproduction: fork-aware citation recovery retrieving component-level code at function-level granularity, Graph-of-Thought synthesis in topological dependency order, RAG-guided in-context example selection from over 20 verified implementations, and visual feedback combining PSNR-guided regeneration, Gaussian-level structural checks, and VLM-driven patching. (3) Knowledge-driven compositional improvement that autonomously finds weaknesses and composes complementary regularizers, losses, and densification strategies to improve upon original results. (4) Interdisciplinary method discovery where agents retrieve physical priors from outside the 3DGS literature and compose them with rendering knowledge to produce methods for previously unaddressed scene types. (5) SPLATIFY-Bench, an evaluation framework across 30 diverse 3DGS papers. On papers without public code, SPLATIFY matches expert implementations while reducing development time from weeks to minutes, and through compositional discovery further improves PSNR by up to 2.4 dB. We additionally demonstrate novel methods for volumetric nebula rendering and other scientific domains, synthesized entirely by SPLATIFY.

Problem

• 3D Gaussian Splatting
• paper-to-code
• reimplementation
• code generation
• 3DGS

Innovation

• 3D Gaussian Splatting
• Multi-agent framework
• Context-free grammar
• Graph-of-Thought
• Compositional discovery