FactorSplat: Appearance-Controllable Gaussian Proxies for Medical Volume Rendering

2026-10-01

AI Summary

This study addresses the limitation of conventional Gaussian proxies in medical volume rendering, where fixed transfer functions hinder flexible appearance adjustment. We propose N-dimensional Gaussian proxies that accept region-specific RGBA curves at inference time. By integrating local lookup with residual learning, our method achieves controllable rendering and, for the first time, enables unseen editing combinations and out-of-distribution modifications from a single checkpoint without retraining, while preserving geometry and directional appearance sharing. Technically, it incorporates low-rank factorization, a shared functional encoder, transfer-function-aware pruning, and a cache-accelerated renderer. Experiments demonstrate PSNR improvements of 1.10–1.52 dB, real-time performance at 524 FPS for 1600×1600 resolution, and an appearance-switching latency of merely 1.17 ms.

Abstract

Transfer functions (TFs) control color and visibility in medical volume rendering, but image-trained Gaussian proxies typically bake one transfer function into their appearance. We present FactorSplat, a per-scene N-dimensional Gaussian splatting (N-DGS) proxy that accepts region-specific intensity-to-RGBA curves at inference. A local lookup applies the authored color and opacity change, while a shared functional encoder and low-rank per-Gaussian factors learn the residual appearance response. Geometry and directional appearance remain shared across presets, with visibility control and TF-aware pruning preserving the ability to hide and reveal structures. On seven CT and MR scans, FactorSplat improves mean PSNR and changed-region error over region-aware VEG across validation, interpolation, unseen composition, and out-of-distribution (OOD) edits. Across these four splits, seven-scan mean PSNR gains over VEG range from 1.10 to 1.52 dB. One checkpoint per scan supports unseen edits without retraining. At $1600^2$, the cached fast renderer averages 524 FPS with 1.17 ms TF switches. Project page: https://gaozhongpai.github.io/FactorSplat/.

Problem

• Medical Volume Rendering
• Gaussian Splatting
• Transfer Function
• Appearance Control

Innovation

• Gaussian Splatting
• Medical Volume Rendering
• Transfer Function
• Low-Rank Factorization
• Appearance Control