🤖 AI Summary
Converting ROS bags into machine learning datasets often relies on ad hoc scripts, resulting in substantial engineering overhead and inefficient iteration. This work introduces, for the first time, the principles of software build systems to robotic dataset construction, proposing a reproducible, incremental generation method grounded in artifact- and dependency-graph semantics. We present Bagzel, an open-source tool built on Bazel, which supports export to the nuScenes format and incorporates Bagzel-xattr for server-side metadata management. Experimental evaluation demonstrates that, on a 20.4 GB dataset, hot builds achieve up to a 386.26× speedup and incremental builds are accelerated by 7.21×, with performance gains further amplified as dataset scale increases.
📝 Abstract
Robotic systems generate large volumes of multimodal sensor data, but converting ROS bag recordings into machine learning datasets is often handled by ad hoc sequential scripts, creating engineering overhead and slow iteration cycles. We model dataset construction as an artifact-based build process over a dependency graph and implement this approach in Bagzel, an open-source Bazel extension for reproducible, incremental dataset generation (including nuScenes-format export). We compare Bagzel and Bagzel-xattr (server-side digest management) against a sequential rosbag2nuscenes baseline. Bagzel reduces runtime in all evaluated execution modes, with the largest gains in iterative workflows (up to 386.26x in warm builds and 7.21x in incremental builds on a 20.4 GB dataset). Across dataset sizes from 5.1 to 20.4 GB, Bagzel variants show markedly better scaling behavior than the baseline, especially in warm and incremental modes. Bagzel-xattr provides additional gains, with a mean runtime reduction of 5.9% compared to Bagzel in the input granularity study. Overall, modeling robotics dataset construction as an artifact-based build process substantially reduces dataset update latency while maintaining a deterministic build design that supports reproducibility. Bagzel is publicly available at https://github.com/UniBwTAS/bagzel.