Beyond Monoscopic Viewing: A Study on 3D Gaussian Splatting Quality in VR

2026-09-29

AI Summary

This study addresses the limitation that existing monocular evaluation metrics severely underestimate local artifacts in 3D Gaussian Splatting (3DGS) reconstructions for virtual reality (VR) scenes, failing to capture depth misalignments under stereoscopic vision. To overcome this, we conduct head-mounted display-based stereoscopic rendering and user studies comparing Structure-from-Motion (SfM) and VGGT initialization schemes to systematically evaluate the perceptual quality of 3DGS in VR. This work reveals the failure mechanisms of standard image metrics under stereoscopic conditions and establishes the necessity of stereoscopic evaluation for VR quality assessment. Experimental results demonstrate a 78.2% user preference rate under stereoscopic viewing, confirming that monocular metrics significantly underestimate perceptual artifacts in VR environments.

Abstract

Stereoscopy is fundamental to virtual reality (VR), providing depth perception through binocular viewing. Recent advances in 3D Gaussian Splatting (3DGS) enable high-quality novel view synthesis, making it well suited to immersive VR. We render 3DGS reconstructions stereoscopically and evaluate them in a head-mounted display, replicating how they would actually be viewed in VR. Real-world capture provides only a limited number of views, and under this constraint 3DGS reconstruction often produces localized floaters and misplaced structures. Standard metrics miss these localized artifacts, which become salient under stereoscopic viewing, where geometry is placed at the wrong depth. We investigate whether standard image-quality evaluation reflects the perceptual quality of 3DGS reconstructions under reduced capture. We compare SfM-only baseline with a union initialization that combines SfM with a dense VGGT network. All other training components are held fixed, isolating the effect of initialization coverage. We conduct a user study comparing preferences under monoscopic and stereoscopic HMD viewing, and test whether image-quality metrics predict the observed preferences. Monoscopically, preference for the more consistent reconstruction is weak, reaching 58.4\% overall. Stereoscopically, the same preference rises to 78.2\% and is consistent across all participants, while image-quality metrics (PSNR, SSIM and LPIPS) and stereo-aware metrics (iSQoe and StereoQA) show only modest differences and fail to penalize them. Our results indicate that monoscopic evaluation and standard image-quality metrics substantially underestimate perceptual artifacts observed in 3DGS reconstructions for VR, making stereoscopic assessment essential for 3DGS quality evaluation in VR.

Problem

• 3D Gaussian Splatting
• Virtual Reality
• Stereoscopic Viewing
• Quality Assessment
• Perceptual Artifacts

Innovation

• 3D Gaussian Splatting
• Virtual Reality
• Stereoscopic Evaluation
• Image Quality Metrics
• Joint Initialization