Liaohe-CobotMagic-PnP: an Imitation Learning Dataset of Intelligent Robot for Industrial Applications

2025-09-27

AI Summary

In Industry 4.0, dynamic environmental disturbances induce strong nonlinear coupling between robot states and behaviors, posing significant challenges for robust perception and control. Method: This work introduces the first industrial-grade multimodal disturbance dataset tailored for complex operational scenarios. It innovatively models multidimensional disturbances—including object size, color, and illumination variations—and designs high-difficulty scenes with geometric similarity >85% and standardized illumination gradients. Microsecond-precision time synchronization and vibration-resistant acquisition protocols ensure data fidelity and task rigor. Sensor fusion—integrating vision, joint torque, and robot state—is implemented via ROS for high-accuracy, multi-source data collection, enabling imitation learning and robust control algorithm development. Contribution/Results: Experiments demonstrate substantial improvements in model operational stability and generalization robustness under dynamic disturbances. The dataset is publicly released to support both academic research and industrial applications.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Security and Privacy: Data transparency and provenance
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

In Industry 4.0 applications, dynamic environmental interference induces highly nonlinear and strongly coupled interactions between the environmental state and robotic behavior. Effectively representing dynamic environmental states through multimodal sensor data fusion remains a critical challenge in current robotic datasets. To address this, an industrial-grade multimodal interference dataset is presented, designed for robotic perception and control under complex conditions. The dataset integrates multi-dimensional interference features including size, color, and lighting variations, and employs high-precision sensors to synchronously collect visual, torque, and joint-state measurements. Scenarios with geometric similarity exceeding 85% and standardized lighting gradients are included to ensure real-world representativeness. Microsecond-level time-synchronization and vibration-resistant data acquisition protocols, implemented via the Robot Operating System (ROS), guarantee temporal and operational fidelity. Experimental results demonstrate that the dataset enhances model validation robustness and improves robotic operational stability in dynamic, interference-rich environments. The dataset is publicly available at:https://modelscope.cn/datasets/Liaoh_LAB/Liaohe-CobotMagic-PnP.

Problem

• Addresses multimodal sensor fusion for dynamic environments
• Improves robotic perception and control under interference
• Enhances model robustness in industrial automation scenarios

Innovation

• Multimodal sensor fusion for dynamic environments
• High-precision synchronized visual-torque-joint data
• ROS-based microsecond-level time-synchronization protocol