Relevance-driven Decision Making for Safer and More Efficient Human Robot Collaboration

2024-09-21

arXiv.org

AI Summary

To address the fundamental trade-off between safety and efficiency in human–robot collaboration (HRC), this paper introduces a novel paradigm centered on “semantic relevance” as a cognitive guidance dimension. Methodologically, we propose a real-time–asynchronous dual-loop decision framework: the asynchronous loop leverages large language models (LLMs) to inject world knowledge and quantify semantic relevance among multimodal scene elements; the real-time loop dynamically drives human intention prediction, task allocation, and collision-avoidant motion planning based on relevance scores. Key contributions include: (1) the first formalization of semantic relevance as a cognitive modeling dimension for HRC; (2) the first dual-loop architecture integrating LLM priors with real-time perception; and (3) relevance-driven dynamic task allocation and prediction-augmented collision avoidance. Experiments demonstrate an intention prediction accuracy of 0.90 and relevance quantification accuracy of 0.96; compared to state-of-the-art methods, our approach reduces collision incidents by 63.76% and collision frames by 44.74%.

Technology Category

• Intelligent Robots: Human-Robot Interaction
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Natural Language Processing: Safety and Robustness

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Human brain possesses the ability to effectively focus on important environmental components, which enhances perception, learning, reasoning, and decision-making. Inspired by this cognitive mechanism, we introduced a novel concept termed relevance for Human-Robot Collaboration (HRC). Relevance is a dimensionality reduction process that incorporates a continuously operating perception module, evaluates cue sufficiency within the scene, and applies a flexible formulation and computation framework. In this paper, we present an enhanced two-loop framework that integrates real-time and asynchronous processing to quantify relevance and leverage it for safer and more efficient human-robot collaboration (HRC). The two-loop framework integrates an asynchronous loop, which leverages LLM world knowledge to quantify relevance, and a real-time loop, which performs scene understanding, human intent prediction, and decision-making based on relevance. HRC decision-making is enhanced by a relevance-based task allocation method, as well as a motion generation and collision avoidance approach that incorporates human trajectory prediction. Simulations and experiments show that our methodology for relevance quantification can accurately and robustly predict the human objective and relevance, with an average accuracy of up to 0.90 for objective prediction and up to 0.96 for relevance prediction. Moreover, our motion generation methodology reduces collision cases by 63.76% and collision frames by 44.74% when compared with a state-of-the-art (SOTA) collision avoidance method. Our framework and methodologies, with relevance, guide the robot on how to best assist humans and generate safer and more efficient actions for HRC.

Problem

• Enhance HRC safety via relevance-driven decision-making
• Improve HRC efficiency using real-time relevance quantification
• Reduce collisions via relevance-based motion generation

Innovation

• Relevance-driven two-loop HRC framework
• LLM-enhanced asynchronous relevance quantification
• Human trajectory-based collision avoidance