Fast Wave-optics Rendering of Multiplane Images for 3D Holographic Displays

2026-07-21

AI Summary

Existing neural rendering methods struggle to adapt to holographic displays and often lack realistic immersion. To address this limitation, this work proposes a wave-optics-based rendering pipeline leveraging multi-plane images (MPI), which for the first time integrates MPI representations with a wave-optical model to efficiently generate computer-generated holograms (CGHs). By employing fast Fourier transforms for wavefront propagation, the method achieves high-fidelity imaging while accelerating computation by up to 250,000× compared to conventional layered CGH algorithms. Experimental results demonstrate that the approach efficiently reconstructs high-quality 3D focal stacks and 4D light fields across multiple 3D datasets, significantly outperforming current state-of-the-art techniques.

Abstract

Recent advances in neural rendering have unlocked unprecedented capabilities in 3D reconstruction and novel view synthesis, giving rise to applications such as virtual fly-throughs of a 3D scene reconstructed from a set of sparse, casually captured images. However, these renderings are viewed on a computer screen or conventional VR headsets as 2D images, greatly limiting the perceptual realism and immersiveness of such experiences. The rapid development in novel 3D scene representations calls for dedicated rendering algorithms that convert these readily-available 3D contents into formats that are compatible with emerging 3D display technologies, such as holographic displays. In this paper, we propose a wave-optics rendering pipeline that works with multiplane images (MPIs) for efficient and high-quality hologram synthesis. Our MPI-based computer-generated holography algorithm greatly outperforms state-of-the-art primitive-based CGH algorithms in terms of runtime, achieving speedups up to 250,000x while achieving comparable image quality, and significantly outperforms conventional layer-based CGH algorithms in terms of image quality. We validate our method extensively on a wide variety of 3D scene datasets both in simulation and through experimentally captured results, showing exceptional 3D focal stack and 4D light field reconstruction performance without sacrificing efficiency.

Problem

• holographic displays
• 3D rendering
• multiplane images
• computer-generated holography
• wave-optics rendering

Innovation

• multiplane images
• computer-generated holography
• wave-optics rendering
• 3D holographic displays
• neural rendering