A Differentiable Optimization Framework for Registering Sequential Bounding Boxes with Point Cloud Stream

2026-09-28

AI Summary

This study addresses the inherent trade-off between geometric accuracy and temporal smoothness in 3D bounding box registration by proposing a training-free, end-to-end joint optimization framework. The method directly incorporates temporal smoothness constraints into the registration objective function, leveraging differentiable optimization and the L-BFGS algorithm to globally solve for all poses within LiDAR point cloud streams. This formulation entirely eliminates conventional post-hoc smoothing strategies. Furthermore, it introduces a ground-truth-free adaptive rule for temporal scale selection. Experimental results demonstrate that, under sufficient observation conditions, the proposed approach significantly reduces trajectory roughness while improving Intersection over Union (IoU). Additionally, this work explicitly delineates the performance boundaries imposed by visibility conditions.

Abstract

Refining a sequence of coarse 3D bounding boxes against a LiDAR point-cloud stream demands tracks that are geometrically accurate (high IoU) and temporally coherent (low roughness), preferably without training data. The usual recipe keeps the two concerns apart: register each frame independently, then smooth the trajectory afterwards with a Kalman~RTS or Savitzky--Golay filter. Smoothing displaces boxes from a geometric optimum and never re-optimises, so it trades accuracy for smoothness. We instead fold the temporal smoothness constraint into a training-free registration objective and solve for all poses jointly with L-BFGS. The payoff depends on how well the object is seen. On well-observed tracks it is large: within the low-roughness budget, the joint objective beats both post-hoc smoothers on paired multi-seed statistics and cuts roughness several-fold relative to frame-wise registration at matched accuracy. Treating visibility as an experimental variable exposes the limit. The advantage decays monotonically as views become one-sided, until it is indistinguishable from zero for near-edge-on objects and slightly negative under a ray-cast simulator with range-dependent density and ego motion, where the decoupled pipeline is in fact ahead at tight roughness budgets. We locate that boundary and trace it to one term: orientation alignment ties yaw to the estimated velocity and fails once that estimate is noisy. A ground-truth-free rule can choose the temporal scale and keep every track inside the roughness budget.

Problem

• 3D bounding box registration
• point cloud stream
• temporal coherence
• geometric accuracy
• tracking refinement

Innovation

• Differentiable Optimization
• Point Cloud Registration
• Joint Pose Estimation
• Temporal Smoothness
• Training-free