RayOcc: Occlusion-Aware Ray Occupancy Estimation via Gaussian Mixture Intensity

2026-07-20

AI Summary

This work addresses the challenge of modeling multiple surface hypotheses along a single ray in monocular 3D semantic occupancy prediction, which arises due to depth ambiguity and occlusion. The authors propose a novel approach that reformulates ray modeling as a multi-label existence prediction problem, uniquely integrating unnormalized Gaussian mixture intensities with a Poisson event model to produce interval-based occupancy probabilities that permit multiple coexisting occupancy hypotheses per ray. The method initializes and rasterizes sparse 3D Gaussian primitives and fuses multi-view image cues for enhanced geometric reasoning. Evaluated on the nuScenes benchmark, the proposed approach outperforms existing Gaussian-based occupancy prediction methods, achieving state-of-the-art performance in both overall IoU and mean IoU metrics.

Abstract

Camera-only 3D semantic occupancy prediction aims to infer voxel-wise scene semantics from multi-view images, yet remains fundamentally challenging due to depth ambiguity and occlusion. Along a single camera ray, multiple spatially separated surfaces may coexist, making occupancy inherently a multi-label existence problem rather than a single-depth estimation task. However, most existing approaches favor a single dominant depth hypothesis per ray, limiting their ability to model volumetric scenes under complex occlusion. To address this limitation, we introduce RayOcc, an occlusion-aware ray occupancy framework that reformulates ray modeling as multi-label existence prediction. Instead of predicting a categorical depth distribution, RayOcc estimates a non-normalized Gaussian mixture intensity along each ray and converts it into interval-wise occupancy probabilities via a Poisson event formulation, allowing multiple occupied hypotheses to coexist without enforcing mutual competition across depth. The predicted mixture components are interpreted as occupancy hypotheses to initialize sparse 3D Gaussian primitives, which are refined and rasterized for semantic occupancy prediction. Experiments on the nuScenes benchmark show that RayOcc achieves state-of-the-art overall IoU and mIoU among the compared Gaussian-based occupancy methods.

Problem

• 3D semantic occupancy
• occlusion
• depth ambiguity
• multi-label existence
• camera-only

Innovation

• Ray Occupancy
• Gaussian Mixture
• Occlusion-Aware
• Multi-label Existence
• Semantic Occupancy Prediction