PIE-PS: Photometric Stereo from Physical Irradiance Event Streams

2026-10-06

AI Summary

This study addresses the challenge of accurate surface normal reconstruction in event-based photometric stereo, where unknown thresholds and data sparsity impose significant limitations. To overcome these issues, we propose an end-to-end framework based on physical irradiance events. Specifically, we first define a threshold-free physical irradiance feature representation. We then employ graph neural networks to model spatiotemporal context and introduce a reliability-graded attention mechanism for adaptive feature aggregation. Finally, dense normal estimation is achieved through a pixel-wise aggregation strategy. Extensive experiments on both synthetic and real-world datasets demonstrate that the proposed method significantly outperforms existing approaches, effectively enhancing the accuracy and robustness of surface normal reconstruction.

Abstract

Event cameras record asynchronous log-image-irradiance changes with microsecond latency and high dynamic range. These properties are useful for photometric stereo under moving illumination, but raw events are sparse and depend on an unknown contrast threshold. We start from the event trigger model and derive a physical relation between adjacent events, light motion, and surface normals. This relation gives a direct physics-only solver, but the solver needs the threshold, enough events at each pixel, and independent per-pixel optimization. To address these limits, we introduce PIE-PS, a learning-based framework for dense surface normal reconstruction from raw event streams and known lighting. We form Physical Irradiance Events (PIEs) by pairing two adjacent events at the same pixel with their corresponding light directions. Each PIE provides a Physical Irradiance Event Feature (PIEF), defined as the signed event rate. PIEF does not require the unknown contrast threshold. To share spatial and temporal context across nearby PIEs, we introduce PIE-GNN, which treats each PIE as a graph node and encodes it with its light-pair geometry. Since the reliability of PIE observations can vary with local appearance, illumination geometry, and sensor noise, Reliability-Grading Attention (RGA) predicts reliability weights to down-weight unreliable PIEs. Pixel aggregation then produces dense normals. Experiments on synthetic and real data show that PIE-PS outperforms prior event-based photometric stereo methods and the direct solver baseline.

Problem

• Photometric Stereo
• Event Camera
• Surface Normal Reconstruction
• Moving Illumination

Innovation

• Photometric Stereo
• Event Camera
• Graph Neural Network
• Physical Irradiance Event
• Reliability-Grading Attention