implement data synchronization

High-precision online estimation algorithms for robotics are highly sensitive to sensor timestamp accuracy; however, existing synchronization solutions struggle to simultaneously achieve real-time operation, low cost, and high temporal precision. To address this, we propose a real-time, trigger-based time synchronization system built on commodity hardware. Our approach employs a hardware-triggered mechanism to jointly schedule heterogeneous sensors operating at different frequencies, and integrates an enhanced clock synchronization protocol with nanosecond-resolution timestamping to ensure precise coordination between sensors and the onboard computer. Crucially, the system eliminates reliance on expensive dedicated timing hardware, thereby substantially mitigating the impact of timing errors on online estimation. Experimental evaluation on a physical robot platform demonstrates sub-microsecond synchronization accuracy, along with significant improvements in both estimation robustness and real-time performance.

Existing systems struggle to support strict audio-visual synchronization, limiting the analysis of fine-grained temporal features in dialogue such as turn-taking, overlapping speech, and prosody. To address this challenge, this work proposes an end-to-end multimodal acquisition and calibration framework that treats synchronized audio and video as equally central modalities for the first time. By integrating a multi-camera array with multi-channel microphones under a unified temporal architecture, the system enables scalable, reproducible, high-quality recording. Standardized calibration and quality control procedures ensure high temporal consistency across modalities, yielding data that effectively supports fine-grained analysis of conversational behavior and data-driven modeling.

This work addresses the challenge of time synchronization among heterogeneous sensors in roadside and vehicle-mounted multi-LiDAR–multi-camera systems by proposing an open-source, modular, and scalable hardware synchronization solution. Using the LiDAR synchronization pulse as a reference, the system employs programmable delay circuits to generate independent trigger signals for each camera, enabling flexible and precise spatiotemporal alignment. The architecture supports arbitrary combinations of sensor counts and has been validated on both a three-camera roadside platform and a seven-camera vehicular setup. Experimental results demonstrate significantly improved spatial consistency between point clouds and images, while the design ensures robustness, reproducibility, and ease of deployment.

Heterogeneous camera systems—e.g., visible-light/infrared, professional/consumer-grade, or audio-equipped/audio-less setups—lack hardware synchronization in real-world scenarios, leading to significant spatiotemporal misalignment across multi-view videos. Method: This paper proposes a vision-based time-encoding method leveraging a custom-designed LED Clock. By embedding temporal exposure timestamps within frames using red and infrared LEDs, the approach achieves cross-modal, audio-free, and external-timecode-free millisecond-level synchronization. It further integrates RMSE-optimized temporal alignment with joint multi-device calibration. Contribution/Results: The method reduces synchronization residuals to 1.34 ms—substantially outperforming existing optical signal, audio-based, and timecode synchronization schemes. Validated in large-scale surgical recordings involving over 25 heterogeneous cameras, it significantly improves downstream tasks including multi-view 3D reconstruction and pose estimation.

Current evaluation methods for audio-visual talking head generation rely on frame-level metrics that assume strict temporal alignment between generated and reference videos, rendering them sensitive to natural variations in speech rate, rhythm, and stylistic expression, and thereby introducing assessment bias. This work reframes evaluation as a sequence alignment problem and introduces Soft Dynamic Time Warping (Soft DTW) to align feature trajectories temporally, enhancing robustness to timing offsets while preserving sequential constraints. The proposed unified sequence-level evaluation framework subsumes frame-level metrics as a special case of rigid alignment, enabling compatibility with existing perceptual, identity, and synchronization encoders without modification. Large-scale experiments across 20 methods and 7 datasets demonstrate that the approach yields more stable evaluations with higher cross-dataset consistency, clearly disentangling trade-offs between synchronization and realism, as well as expressiveness and stability.

Existing video–audio joint generation methods perform well at the semantic level but still struggle with fine-grained temporal synchronization—specifically, the precise alignment of audio events with their visual triggers. This work proposes SyncDPO, a novel framework that integrates Direct Preference Optimization (DPO) with rule-driven temporal perturbation to construct negative samples without additional sampling or annotations, combined with a curriculum learning strategy that progressively refines the model’s ability to discriminate temporal misalignments from coarse to fine granularity. Evaluated on four diverse benchmarks, SyncDPO significantly outperforms prior approaches, demonstrating superior temporal alignment and stronger out-of-distribution generalization in both objective metrics and subjective evaluations.

This work addresses the limited parallelizability of the classical dynamic time warping (DTW) algorithm, which suffers from quadratic time and memory complexity. The authors propose Segmental DTW, a novel approach that decomposes global sequence alignment into local subsequence DTW computations that can be executed in parallel, followed by a segment-level dynamic programming step to integrate the partial alignments. This method achieves near-full parallelism while preserving alignment accuracy comparable to standard DTW. Theoretical analysis and empirical evaluation on Chopin Mazurka audio alignment tasks demonstrate that one variant of the proposed method outperforms existing approaches in both computational efficiency and alignment performance.

This work addresses the limitations of existing audio-visual synchronization evaluation methods, which struggle to disentangle temporal alignment from semantic consistency and suffer from coupling biases in data construction. We propose the first structured and scalable benchmark framework that enables independent assessment of temporal synchronization and semantic correspondence. Through a hybrid pipeline combining automated filtering and human verification, we construct a large-scale dataset comprising 3,269 videos and 38,390 samples across three audio categories—speech, music, and environmental sounds—and ten diverse scenarios. The dataset ensures authentic on-screen sound sources and supports both multimodal alignment analysis and downstream task evaluation. Using this benchmark, we systematically evaluate five representative models. Both code and data are publicly released.