Vision Language Model-based Testing of Industrial Autonomous Mobile Robots
🤖 AI Summary
Industrial autonomous mobile robots (AMRs) face high-cost, high-risk, and low-coverage safety testing in human-shared environments. To address this, we propose RVSG—a novel method that introduces vision-language models (VLMs) into AMR testing for the first time, enabling automated synthesis of semantically guided, safety-violating human behavioral scenarios from natural-language functional and safety requirements. RVSG leverages VLMs to interpret requirements, generate diverse human–robot interaction perturbations, and validate them in closed-loop simulation with a PAL Robotics AMR. Experimental results demonstrate that RVSG significantly improves both efficiency and diversity in generating requirement-violating scenarios: robot behavioral variability increases by 37%, and defect detection rate rises by 2.1× compared to baseline methods. Crucially, RVSG effectively uncovers previously unknown safety vulnerabilities arising from unanticipated human–robot interactions.
Application Category
📝 Abstract
Autonomous Mobile Robots (AMRs) are deployed in diverse environments (e.g., warehouses, retail spaces, and offices), where they work alongside humans. Given that human behavior can be unpredictable and that AMRs may not have been trained to handle all possible unknown and uncertain behaviors, it is important to test AMRs under a wide range of human interactions to ensure their safe behavior. Moreover, testing in real environments with actual AMRs and humans is often costly, impractical, and potentially hazardous (e.g., it could result in human injury). To this end, we propose a Vision Language Model (VLM)-based testing approach (RVSG) for industrial AMRs developed by PAL Robotics in Spain. Based on the functional and safety requirements, RVSG uses the VLM to generate diverse human behaviors that violate these requirements. We evaluated RVSG with several requirements and navigation routes in a simulator using the latest AMR from PAL Robotics. Our results show that, compared with the baseline, RVSG can effectively generate requirement-violating scenarios. Moreover, RVSG-generated scenarios increase variability in robot behavior, thereby helping reveal their uncertain behaviors.
Problem
Research questions and friction points this paper is trying to address.
Testing AMRs for safe human interaction in diverse environments
Reducing cost and hazard of real-world AMR testing scenarios
Generating diverse human behaviors to reveal robot uncertainties
Innovation
Methods, ideas, or system contributions that make the work stand out.
VLM-based testing for AMRs
Generates diverse human behaviors
Simulator evaluation reveals uncertainties
🔎 Similar Papers
No similar papers found.
