contour-based differentiable rendering

Conventional graphics pipelines impose severe memory and performance bottlenecks in backward gradient computation for 3D Gaussian Splatting (3DGS). Method: We propose the first differentiable hardware rasterization scheme tailored for 3DGS, featuring a programmable hybrid blending rasterizer that supports gradient propagation. Our architecture integrates quad-level and subgroup-level gradient reduction, employs tile-free splat processing, optimizes atomic operations, and introduces subgroup-level parallel gradient computation. We further conduct the first systematic evaluation of mixed-precision rendering using FP16 and UNORM16, identifying their optimal trade-off between accuracy and speed. Contribution/Results: Experiments show our backward rasterizer achieves >10× speedup over naive atomic-based methods and 3× over classical tile-based approaches. End-to-end training accelerates by 3.07×, while memory overhead amounts to only 2.67% of splat sorting memory consumption.

This work proposes Gaussian Mesh Rendering (GMR), a novel differentiable rendering framework that integrates the efficient rasterization mechanism of 3D Gaussian splatting with triangle mesh representations. Traditional mesh-based differentiable renderers suffer from high computational costs and non-smooth gradients, hindering efficient optimization under limited memory. GMR addresses these limitations by analytically generating Gaussian primitives for each triangular face, yielding a lightweight, structure-aware renderer. This approach preserves geometric fidelity while producing smoother gradients, significantly improving optimization efficiency and reconstruction quality—particularly under small-batch, low-memory conditions.

To address the challenge of jointly rendering RGB images, depth maps, surface normals, and semantic logits while preserving cross-scene geometric consistency in high-fidelity multimodal 3D reconstruction, this paper proposes a geometry-aware unified Gaussian rasterization framework. Our method introduces two key innovations: (1) a differentiable ray-ellipsoid intersection renderer that analytically derives gradients for depth and normal predictions, enabling joint optimization of Gaussian rotation and scale to enhance geometric fidelity; and (2) a CUDA-accelerated differentiable rasterization pipeline integrating learnable attributes and a differentiable pruning mechanism to balance efficiency and representational capacity. Evaluated on multiple benchmarks, our approach achieves state-of-the-art performance, significantly improving multimodal reconstruction quality and cross-view geometric consistency.

Differentiable ray casting on irregularly distributed Gaussian kernels suffers from severe rendering artifacts (e.g., splatter) and low efficiency in novel-view synthesis. Method: We propose a voxelized Gaussian modeling framework that establishes, for the first time, a physically consistent, density-radiance decoupled differentiable ray casting model. Our approach introduces hierarchical voxel slab integration with BVH acceleration for efficient and accurate volumetric rendering; jointly represents full-spectrum color using spherical Gaussians and spherical harmonics; and co-optimizes Gaussian geometry and radiance properties. Contribution/Results: On the Blender dataset, our method achieves real-time inference at 25 FPS with reasonable training efficiency. It significantly outperforms state-of-the-art methods in PSNR and SSIM while effectively suppressing splatter artifacts. This work bridges a critical gap between differentiable ray casting and Gaussian-based scene representation.

Existing video prediction models operating in discrete pixel space often rely on mean squared error (MSE) loss, which tends to produce overly smoothed predictions with diminished detail. To address this limitation, this work proposes a novel Predictive Differentiable Rendering (PDR) paradigm that introduces 2D Gaussian differentiable rendering into video prediction for the first time. The approach balances continuous representation and discrete prediction through a lightweight, plug-and-play adapter, PredGS, coupled with an efficient CUDA-accelerated renderer, predgsplat. By jointly optimizing L1 loss and structural similarity (SSIM), the method significantly enhances visual fidelity, detail preservation, and prediction accuracy across multiple benchmarks—including TaxiBJ, WeatherBench, KTH, and Human3.6M—while achieving up to a 10× speedup in rendering compared to baseline methods.

This work addresses the limitation of existing differentiable renderers, which are largely confined to vector graphics and struggle to efficiently handle real-world bitmap images. We present the first efficient differentiable rendering engine capable of optimizing large-scale bitmap primitives. Our approach leverages a custom CUDA implementation for parallel gradient computation, combined with Gaussian-blurred soft rasterization, structure-aware initialization, noise-based canvases, and specialized loss functions incorporating video and spatiotemporal constraints. The system jointly optimizes the position, rotation, scale, color, and opacity of thousands of bitmaps in under one minute on a consumer-grade GPU, substantially improving both optimization efficiency and image fidelity. An open-source Python package is released to support integration into creative workflows and layered file export, thereby advancing the practical adoption of differentiable rendering in real-world artistic applications.

This work proposes a differentiable 3D representation that unifies real-time ray tracing and rasterization by addressing a key limitation of existing foam-based representations: their unbounded cells hinder tile-based rasterization. To resolve this, we replace traditional Voronoi foams with bounded power diagrams, effectively constraining cell extents while preserving efficient ray traversal. We further introduce an oriented explicit surface to model the interface between interior and exterior regions and embed differentiable textures to decouple geometry from appearance. This approach generalizes foam structures into controllable, bounded power diagrams without requiring expensive Delaunay triangulation during training. The resulting method matches the rasterization performance of 3D Gaussian Splatting (3DGS) while maintaining state-of-the-art ray tracing efficiency, offering a unified and practical framework for real-time differentiable rendering.

This work addresses the challenge of balancing rendering speed, model size, and performance under sparse input views—a key limitation for deploying novel view synthesis methods on resource-constrained devices. The authors propose a new approach based on Multi-Plane Image (MPI) representation, leveraging depth maps predicted by vision foundation models for geometric initialization. A one-step diffusion mechanism is introduced to jointly optimize the MPI representation in a differentiable manner and enhance the final rendered output. This design significantly improves scene completeness and visual fidelity from sparse views. Compared to representative 3D Gaussian splatting methods, the proposed method achieves a 30.7% faster inference speed and reduces model size to only 14.8% of the baseline, while delivering competitive synthesis quality in forward-facing scenes.

本文通过优化场景表示、GPU执行等方法，解决了光线追踪在实时渲染中速度慢的问题，实现了比现有方法更快的渲染速度。

This work addresses the gradient instability and limited expressiveness of planar rational splines—such as NURBS—in differentiable rendering by introducing a continuous Gaussian field–based differentiable vector rendering framework. The method reformulates rendering as a smooth, differentiable accumulation process by sampling Gaussian kernels over both the curve parameter domain and the interior of closed regions. It is the first approach to jointly support long splines, rational weights, non-uniform knot vectors, and filled closed regions within a unified differentiable pipeline. Compared to conventional analytical rasterization techniques, the proposed framework substantially enhances gradient stability and geometric expressiveness, achieving superior reconstruction quality and robustness in tasks including calligraphy reconstruction, vectorization, and image abstraction with long splines.