video-to-event conversion

Existing RGB-based novel view synthesis and 4D reconstruction methods suffer from limited performance under low-light conditions and high-speed motion—due to illumination dependency, motion blur, and narrow dynamic range. To address this, we propose the first event-driven dynamic Gaussian splatting framework. Our method introduces event-stream-guided 3D initialization, adaptive temporal slicing for point cloud construction, and intensity-aware importance pruning to enhance temporal modeling fidelity and 3D consistency. Furthermore, it integrates multi-view event alignment, contrast-adaptive threshold optimization, and native event-based rendering for end-to-end high-fidelity dynamic scene reconstruction. Evaluated on a newly constructed multi-view event dataset, our approach significantly outperforms state-of-the-art pure-event and event-RGB fusion methods, particularly in challenging high-speed motion and low-light scenarios—achieving substantial gains in reconstruction accuracy and geometric fidelity.

High-cost, low-resolution, and autofocus-free conventional event cameras hinder early-stage development and prototyping. This paper proposes a novel low-cost paradigm leveraging standard frame-based cameras to synthesize high-fidelity event streams in real time: raw Bayer data is captured directly—bypassing ISP processing—and fed into an enhanced DVS-Voltmeter model to construct a configurable event simulation framework, enabling, for the first time, synchronized acquisition of raw, RGB, and event modalities. The system incorporates embedded-optimized algorithms, achieving real-time, high-resolution, high-dynamic-range event generation on a Raspberry Pi, with full support for autofocus and flexible parameter tuning. Experimental evaluation demonstrates that the synthesized event streams match or surpass commercial event cameras in spatiotemporal accuracy, dynamic range, and functional completeness. The open-sourced implementation lowers the barrier to entry, accelerating democratization and practical adoption of event-based vision technologies.

Existing synthetic event datasets predominantly rely on dense RGB videos, suffering from limited viewpoint diversity, geometric inconsistency, and high hardware costs. To address these limitations, GS2E introduces the first large-scale, high-fidelity synthetic event dataset generated from sparse multi-view real-world RGB images. It first reconstructs static scenes using 3D Gaussian Splatting, then synthesizes temporally dense, geometrically consistent event streams by integrating adaptive trajectory interpolation with physically grounded contrast-threshold modeling—ensuring robustness to illumination variations and motion dynamics. This approach breaks away from conventional video-based synthesis paradigms, enabling more realistic and scalable event generation. Evaluated on event-driven 3D reconstruction tasks, GS2E significantly improves model generalization across unseen scenes and viewpoints. Moreover, it establishes a new benchmark for event-based vision research, facilitating systematic evaluation of geometry-aware, lighting-robust, and motion-adaptive event processing methods.

To address the degradation in sim-to-real generalization caused by simulation-induced distortions in event camera data, this paper introduces the Event Quality Score (EQS)—the first differentiable and optimization-friendly metric for assessing the realism of event streams. EQS leverages features extracted by the RVT model from both real and synthetic event sequences, jointly quantifying their distributional discrepancy via cosine similarity and Wasserstein distance in the latent space, and maps this discrepancy to a scalar realism score. Systematic evaluation on the DSEC dataset demonstrates that EQS exhibits strong positive correlation (Spearman ρ > 0.92) with real-world performance on downstream detection tasks, significantly outperforming existing hand-crafted metrics. This work establishes, for the first time, a quantitative link between event stream realism and cross-domain generalization capability. The open-sourced implementation enables iterative simulator refinement and effectively bridges the event camera simulation-to-reality gap.

To address the challenge of high-quality continuous video capture under high-speed motion—where conventional RGB cameras suffer from bandwidth and dynamic range limitations—this paper proposes a novel video reconstruction framework jointly driven by a single RGB image and an event stream. Methodologically, it introduces the first synergistic framework integrating continuous long-range motion modeling with neural synthesis, featuring a continuous NeRF-style temporal representation and an event-conditioned generative network; hardware synchronization at millisecond precision is achieved via a custom single-lens beamsplitter enabling concurrent RGB and event acquisition. Key contributions include: (i) the E2D2 benchmark—the first evaluation dataset for extreme-compression video reconstruction; (ii) state-of-the-art reconstruction quality, achieving +3.61 dB PSNR and −33% LPIPS improvement over prior methods; and (iii) superior downstream task performance compared to existing baselines.

To address the high energy consumption of RGB video surveillance, this paper introduces a novel task—Image-and-Event-to-Video (IE2Video)—aiming to reconstruct high-fidelity, full-frame videos from sparse RGB keyframes and continuous asynchronous event streams. Methodologically, we pioneer the integration of an event encoder into a pretrained text-to-video diffusion model (LTX), employing Low-Rank Adaptation (LoRA) for efficient fine-tuning—replacing conventional autoregressive architectures (e.g., HyperE2VID). Evaluated across multiple event-camera datasets, our approach reduces LPIPS perceptual distortion by 33% (from 0.422 to 0.283), supports reconstruction of sequences up to 128 frames long (with robust performance at 32–128 frames), and significantly enhances visual quality and cross-dataset generalization. This work establishes a new paradigm for low-power visual perception by synergizing event-based sensing with diffusion-based video generation.

Existing event camera simulators rely on frame-based sequences to infer event timestamps, struggling with fast motion and occlusions, which degrades simulation accuracy. This work proposes a continuous-time event simulator based on dynamic 3D Gaussian splatting that explicitly models per-pixel brightness change rates through a 3D scene representation, enabling precise prediction of threshold-crossing times. The method is the first to generate multiple events within a single rendering step without temporal upsampling. It further incorporates an occlusion-aware adaptive time-stepping scheme and a tile-based arbiter to emulate real sensor bandwidth constraints. Evaluated on RGB–event paired benchmarks, the approach achieves state-of-the-art fidelity in simulated event streams and demonstrates superior transfer performance in downstream tasks.

Existing event-camera methods for novel-view synthesis of non-rigid objects require auxiliary sparse RGB inputs, limiting practical applicability. This work presents the first monocular event-only approach for dynamic non-rigid novel-view synthesis—without any RGB supervision. Our method introduces a differentiable deformable 3D Gaussian point-based explicit representation, jointly optimizing deformation and radiance fields directly from event streams. To guide learning, we design an event-driven binary mask generation mechanism to encode deformation priors, and incorporate a 2D event observation loss to enforce geometry-appearance co-reconstruction. Evaluated on both synthetic and real-world datasets, our approach significantly outperforms prior baselines, producing high-fidelity, temporally consistent novel views. All code, models, and datasets will be publicly released.

This work addresses the limited generalization of event-driven object detection models caused by variations in event camera sensor parameters, which undermines robustness across devices. For the first time, it systematically uncovers the relationship between intrinsic event camera parameters and detection performance. The authors propose a joint distribution training framework that integrates event data modeling with parameter sensitivity analysis to enable adaptive inference across diverse sensors. By explicitly accounting for sensor-specific characteristics during training, the method significantly improves detection accuracy on unseen event cameras, demonstrating the efficacy of parameter-aware learning for cross-sensor transfer. This advancement represents a critical step toward sensor-agnostic event-based perception.

Video frame prediction in dynamic scenes often suffers from blur and artifacts due to missing information. To address this challenge specifically for event camera data, we propose a diffusion-based framework trained on residual representations for high-fidelity, temporally consistent single-frame synthesis. Our key contributions are: (1) the first event-driven residual-aligned VAE, which implicitly encodes event streams as structured residuals—bypassing explicit optical flow estimation and pixel-level warping; (2) a Diverse-Length Temporal (DLT) augmentation strategy that enhances robustness to varying motion dynamics; and (3) a conditional diffusion denoising module jointly optimizing reconstruction quality and temporal coherence. Extensive evaluations on multiple benchmarks demonstrate substantial improvements over state-of-the-art event- and image-domain frame prediction and interpolation methods. Our synthesized frames exhibit superior sharpness and significantly enhanced temporal consistency.