Representation Handoffs for OpenArm-Based Laboratory Mobile Manipulation

2026-08-07

AI Summary

This work addresses the challenge of aligning natural language instructions with environmental perception to generate safe and executable robotic actions in language-guided laboratory automation. It proposes a “representation handoff” architecture that decouples language understanding, perception, planning, and safety modules, enabling debuggable integration through well-defined intermediate representations while explicitly exposing deployment-critical issues such as calibration, asset management, and visual grounding. Implemented on a dual OpenArm manipulator system mounted on a mobile base and equipped with RGB-D and LiDAR sensors, the framework leverages ROS 2/MoveIt and a configurable skill interface. Through dry runs and startup checks, the system effectively identified and validated key integration bottlenecks, demonstrating the practicality and scalability of the proposed architecture.

Abstract

Open-source robotics and foundation models have lowered the barrier to embodied AI, yet language-guided laboratory automation still requires reliable alignment from instructions and observations to safe actions. This field report presents an OpenArm-based mobile manipulation prototype for laboratory-style tasks, built by integrating dual OpenArm manipulators with a mobile base, vertical slide, RGB-D sensing, lidar-based mapping, ROS2/MoveIt execution, and profile-defined skill interfaces. The system is organized around representation handoffs: natural language requests are constrained into registered skill calls, sensor observations are grounded into maps and object poses, object priors provide role and skill constraints, and runtime bindings compile validated skills into executable motion goals. We use dry-run traces and startup checks to evaluate this integration path, showing how the prototype exposes missing calibration, incomplete object assets, and unfinished real-scene visual grounding as explicit deployment blockers. These intermediate representations serve as practical debugging interfaces for integrating language, perception, planning, and robot safety in embodied systems.

Problem

• representation handoffs
• language-guided automation
• mobile manipulation
• embodied AI
• laboratory robotics

Innovation

• representation handoffs
• mobile manipulation
• language-guided robotics
• embodied AI
• skill interfaces