🤖 AI Summary
To address the efficiency bottleneck of real-time Neural Radiance Fields (NeRF) rendering in large-scale scenes, this paper proposes a hybrid neural representation: a low-polygon base mesh serves as geometric prior, coupled with view-dependent displacement mapping for geometric detail modulation, and integrated with a lightweight compressed NeRF model for appearance modeling. We theoretically and empirically demonstrate—for the first time—that high-fidelity explicit meshes are unnecessary; instead, geometric-appearance decoupling combined with hardware-aware optimization enables high-quality real-time rendering on edge devices. Our method achieves >30 FPS at 1280×720 resolution on a MacBook M1 Pro, improves PSNR by 0.2 dB on the Unbounded-360 indoor dataset, and reduces model size by over 20% compared to state-of-the-art methods. The core contribution is establishing the “light geometry + strong appearance” paradigm, significantly lowering computational and storage overhead, thereby advancing practical on-device deployment of NeRF.
📝 Abstract
Neural Radiance Field (NeRF) has emerged as a leading technique for novel view synthesis, owing to its impressive photorealistic reconstruction and rendering capability. Nevertheless, achieving real-time NeRF rendering in large-scale scenes has presented challenges, often leading to the adoption of either intricate baked mesh representations with a substantial number of triangles or resource-intensive ray marching in baked representations. We challenge these conventions, observing that high-quality geometry, represented by meshes with substantial triangles, is not necessary for achieving photorealistic rendering quality. Consequently, we propose MixRT, a novel NeRF representation that includes a low-quality mesh, a view-dependent displacement map, and a compressed NeRF model. This design effectively harnesses the capabilities of existing graphics hardware, thus enabling real-time NeRF rendering on edge devices. Leveraging a highly-optimized WebGL-based rendering framework, our proposed MixRT attains real-time rendering speeds on edge devices (over 30 FPS at a resolution of $1280 imes 720$ on a MacBook M1 Pro laptop), better rendering quality (0.2 PSNR higher in indoor scenes of the Unbounded-360 datasets), and a smaller storage size (less than 80% compared to state-of-the-art methods).
