TIDES: Time-Derivative Event Simulation via Deformable Reconstruction

2026-06-01

AI Summary

Existing event camera simulators rely on frame-based sequences to infer event timestamps, struggling with fast motion and occlusions, which degrades simulation accuracy. This work proposes a continuous-time event simulator based on dynamic 3D Gaussian splatting that explicitly models per-pixel brightness change rates through a 3D scene representation, enabling precise prediction of threshold-crossing times. The method is the first to generate multiple events within a single rendering step without temporal upsampling. It further incorporates an occlusion-aware adaptive time-stepping scheme and a tile-based arbiter to emulate real sensor bandwidth constraints. Evaluated on RGB–event paired benchmarks, the approach achieves state-of-the-art fidelity in simulated event streams and demonstrates superior transfer performance in downstream tasks.

Abstract

Event cameras emit asynchronous events in response to environmental appearance changes. The scarcity of real-world event datasets makes simulation essential. However, most simulators infer event timestamps from frame sequences, forcing many threshold crossings to share a small set of discrete times; a failure mode we term timestamp batching that worsens under fast motion and occlusion. We present TIDES, a continuous-time event simulator built on dynamic Gaussian splatting. Because TIDES operates on an explicit 3D scene representation with learnt geometry and motion, it can derive per-pixel intensity dynamics directly from the scene, rather than by differencing rendered frames. This enables accurate threshold-crossing prediction, including multiple crossings per rendering step, without temporal upsampling or frame interpolation. The same 3D scene model reveals where objects partially occlude one another; TIDES uses this to guide adaptive time stepping, concentrating computation only in regions where occlusion dynamics make simple models of brightness change unreliable. Finally, we model finite sensor bandwidth using a tile-level arbiter whose throughput, jitter, and event drops reproduce realistic sensor artifacts. Across paired RGB-event benchmarks, TIDES attains state-of-the-art event-stream fidelity. We also show that events simulated by TIDES transfer more effectively to real downstream tasks than competitors'.

Problem

• event camera simulation
• timestamp batching
• occlusion
• fast motion
• event fidelity

Innovation

• event camera simulation
• continuous-time modeling
• dynamic Gaussian splatting
• adaptive time stepping
• sensor bandwidth modeling