D-VLC: Decentralized Vision-Language Collaboration for Heterogeneous Embodied Multi-Robot Systems in Unknown Environments

2026-07-31

AI Summary

This work addresses the challenge of enabling heterogeneous multi-robot systems to efficiently execute complex semantic instructions in unknown environments. Existing approaches are limited by their reliance on predefined models, known maps, and centralized decision-making, resulting in poor generalization. To overcome these limitations, the paper proposes a decentralized asynchronous reasoning framework that leverages lightweight information sharing, capability-aware collaboration, and a unified action interface. This framework enables, for the first time, general-purpose vision-language models (VLMs) to drive semantic coordination among robots without requiring task-specific or robot-specific training. The method operates without prior knowledge of the environment or centralized control, achieving over 70% success rates across diverse scenarios and multiple VLMs, while reducing task completion time by up to 55.8% compared to geometric greedy baselines.

Abstract

Multi-robot systems, particularly heterogeneous robot swarms, can improve the efficiency of complex task execution through parallel collaboration and complementary capabilities. However, conventional rule-based methods rely on predefined task models and specialized decision making programs, making it difficult to understand complex semantic instructions and coordinate heterogeneous robots. LLMs introduce strong language understanding and task reasoning capabilities, allowing multi-robot systems to interpret instructions, decompose tasks, and assign roles according to task semantics. VLMs further incorporate visual perception, enabling robots to reason about objects, regions, and spatial relationships in physical environments. Nevertheless, existing LLM/VLM based methods often depend on known maps, centralized and synchronized decision making, limiting their generalization to heterogeneous robots and unseen tasks. We therefore propose a framework that combines decentralized asynchronous reasoning, lightweight information sharing, capability aware collaboration, and a unified action interface, enabling general purpose VLMs to generate robot specific actions executed by learning free experts without task or robot specific training. Experiments across diverse scenarios and multiple VLMs show success rates above 70\%, with completion time reduced by up to 55.8\% relative to the geometric greedy baseline.

Problem

• heterogeneous multi-robot systems
• unknown environments
• vision-language collaboration
• decentralized coordination
• semantic task understanding

Innovation

• Decentralized reasoning
• Vision-Language Models (VLMs)
• Heterogeneous multi-robot systems
• Capability-aware collaboration
• Learning-free execution