Kleinkram: Open Robotic Data Management

📅 2025-11-25
📈 Citations: 0
✨ Influential: 0
📄 PDF
🤖 AI Summary
To address scalability and collaborative efficiency bottlenecks in storing, indexing, and sharing large-scale unstructured robot data (e.g., ROS bags, MCAP), this paper proposes a modular local-cloud architecture. The architecture integrates S3-compatible object storage, Dockerized processing pipelines, and dual-mode Web/CLI interfaces, introducing the novel “Action Runner” mechanism to automate end-to-end workflows—including data validation, format standardization, metadata indexing, and benchmarking. It enables unified management of heterogeneous robot data formats and has been deployed to reliably host over 30 TB of real-world research data. Empirical evaluation demonstrates significant improvements in data discoverability, reuse rate, and cross-team collaboration. The system establishes a reproducible, scalable infrastructure paradigm for data-intensive robotics research.

Technology Category

Intelligent Robots: Multimodal Perception & Sensor FusionNatural Language Processing: Safety and RobustnessMachine Learning: Time-Series/Data Streams

Application Category

Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applicationsSecurity and Privacy: Data transparency and provenanceSearch and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
📝 Abstract
We introduce Kleinkram, a free and open-source system designed to solve the challenge of managing massive, unstructured robotic datasets. Designed as a modular, on-premises cloud solution, Kleinkram enables scalable storage, indexing, and sharing of datasets, ranging from individual experiments to large-scale research collections. Kleinkram natively integrates with standard formats such as ROS bags and MCAP and utilises S3-compatible storage for flexibility. Beyond storage, Kleinkram features an integrated"Action Runner"that executes customizable Docker-based workflows for data validation, curation, and benchmarking. Kleinkram has successfully managed over 30 TB of data from diverse robotic systems, streamlining the research lifecycle through a modern web interface and a robust Command Line Interface (CLI).
Problem

Research questions and friction points this paper is trying to address.

Managing massive unstructured robotic datasets efficiently
Enabling scalable storage and sharing of robotic data
Executing customizable workflows for data validation and curation
Innovation

Methods, ideas, or system contributions that make the work stand out.

Open-source system for managing robotic datasets
Modular on-premises cloud with scalable storage
Integrated Docker workflows for data processing
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
No related jobs found.
C
Cyrill Puntener
Robotic Systems Lab, ETH Zurich, Switzerland
J
Johann Schwabe
Robotic Systems Lab, ETH Zurich, Switzerland
D
Dominique Garmier
Robotic Systems Lab, ETH Zurich, Switzerland
J
Jonas Frey
Robotic Systems Lab, ETH Zurich, Switzerland; Max Planck Institute for Intelligent Systems, Tübingen, Germany
Marco Hutter
Marco Hutter
Professor of Robotics, ETH Zurich
Legged RoboticsRoboticsControl