Word2Wave: Language Driven Mission Programming for Efficient Subsea Deployments of Marine Robots

2024-09-27

arXiv.org

AI Summary

Low programming efficiency and unnatural human–AUV interaction hinder remote mission deployment for autonomous underwater vehicles (AUVs). Method: We propose a natural language–driven task programming interface comprising: (1) a domain-specific instruction syntax and semantic grammar tailored to underwater operations; (2) a GPT-assisted synthetic data generation pipeline coupled with a lightweight T5-Small end-to-end semantic parsing framework mapping natural language to executable commands; and (3) an interactive interface integrated with 2D geospatial visualization. Contribution/Results: This work pioneers the systematic application of large language model–enhanced few-shot NL2Code to underwater robotics, enabling real-time voice/text-to-command parsing and visual task orchestration. Experiments demonstrate a >90% reduction in task programming time compared to conventional tools and a System Usability Scale (SUS) score of 76.25, significantly improving deployment efficiency and interaction naturalness.

Technology Category

Natural Language Processing: Code Generation / Program Synthesis from Natural LanguagePlanning, Routing, and Scheduling: Planning with Language ModelsIntelligent Robots: Human-Robot Interaction

Application Category

Search and Retrieval-Augmented AI: Natural language understanding for Web searchSemantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactionsSystems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

This paper explores the design and development of a language-based interface for dynamic mission programming of autonomous underwater vehicles (AUVs). The proposed `Word2Wave' (W2W) framework enables interactive programming and parameter configuration of AUVs for remote subsea missions. The W2W framework includes: (i) a set of novel language rules and command structures for efficient language-to-mission mapping; (ii) a GPT-based prompt engineering module for training data generation; (iii) a small language model (SLM)-based sequence-to-sequence learning pipeline for mission command generation from human speech or text; and (iv) a novel user interface for 2D mission map visualization and human-machine interfacing. The proposed learning pipeline adapts an SLM named T5-Small that can learn language-to-mission mapping from processed language data effectively, providing robust and efficient performance. In addition to a benchmark evaluation with state-of-the-art, we conduct a user interaction study to demonstrate the effectiveness of W2W over commercial AUV programming interfaces. Across participants, W2W-based programming required less than 10% time for mission programming compared to traditional interfaces; it is deemed to be a simpler and more natural paradigm for subsea mission programming with a usability score of 76.25. W2W opens up promising future research opportunities on hands-free AUV mission programming for efficient subsea deployments.

Problem

• Develops a language-based interface for AUV mission programming.
• Enables efficient and interactive subsea mission configuration.
• Improves usability and reduces mission programming time significantly.

Innovation

• Language rules for mission mapping
• GPT-based prompt engineering module
• SLM-based command generation pipeline