When Robots Exchange Meaning: A Demo of Goal-Oriented Semantic Communications for Collaborative Robotics

2026-07-30

AI Summary

Traditional communication systems, which prioritize packet-level quality metrics, are ill-suited to meet the demands of collaborative robots requiring semantic understanding and closed-loop task execution. This work proposes a task-driven semantic communication paradigm for 6G-enabled embodied intelligence, featuring an end-to-end experimental platform that integrates visual semantic compression, edge-based semantic mapping, and interactive visualization. Leveraging a VQ-VAE to generate semantic tokens and combining ROS 2, RTAB-Map, and multimodal perception, the system achieves a 42.67× compression ratio (down to 5,400 bytes) on 320×240 images while producing a rich 3D semantic map that fuses depth, pose, and object-level semantics, thereby effectively supporting downstream robotic tasks.

Abstract

Collaborative robotics is a representative task-oriented 6G use-case, where communication quality should be reflected in mission execution, environment understanding, and closed-loop operation rather than packet delivery alone. This demo paper presents a robot-edge semantic communication (SemCom) testbed integrating robot-side visual compression, edge-side semantic mapping, and dashboard-based mission interaction. A mobile robot equipped with RGB-D sensing and LiDAR runs ROS 2, while a Jetson Orin edge node performs reconstruction, RTAB-Map mapping, semantic object handling, and browserbased visualization. As an initial proof of concept, RGB frames are encoded on the robot into VQ-VAE tokens using an ONNX Runtime encoder and reconstructed on the edge using a PyTorch decoder. A 320 X 240 image is represented by an 80 X 60 token grid with a packed payload of 5400 bytes, corresponding to a 42.67X reduction relative to model-input RGB bytes. The reconstructed visual stream is further associated with depth, pose, and 3D mapping information to generate a semantic map for downstream robotic applications. The demo exposes the full path from semantic visual transport to object-level map interaction, and provides a practical platform for future task-aware 6G networking studies at the intersection of SemCom, embodied AI, and physical AI-enabled robotics. A video of the demo is available at https://tinyurl.com/Tos09

Problem

• semantic communication
• collaborative robotics
• task-oriented communication
• 6G
• embodied AI

Innovation

• Semantic Communication
• VQ-VAE
• Collaborative Robotics
• Edge Intelligence
• Task-Oriented Networking