Scenario-Based Field Testing of Drone Missions

2024-07-11

EUROMICRO Conference on Software Engineering and Advanced Applications

AI Summary

In outdoor UAV search-and-rescue testing for aerospace applications, challenges include highly dynamic environments, irreproducible scenarios, and inadequate test guidance. To address these, this paper proposes FiTS (Field Testing Management based on Scenarios), a novel scenario-driven testing methodology. FiTS innovatively integrates scenario-based requirements engineering with Behavior-Driven Development (BDD) to establish a comprehensive test framework supporting dynamic environmental perception, role-based task allocation, structured test design, and iterative optimization. Through formal scenario modeling and role-oriented test specification, FiTS enables adaptive, reusable, and traceable test execution. Empirical validation across three representative search-and-rescue use cases demonstrates that FiTS significantly improves test execution efficiency and data acquisition quality. Furthermore, expert evaluation by three senior developers confirms substantial enhancements in test traceability and analytical capability for post-test data interpretation.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Constraint Satisfaction and Optimization: Search
• Planning, Routing, and Scheduling: Activity and Plan Recognition

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Testing and validating Cyber-Physical Systems (CPSs) in the aerospace domain, such as field testing of drone rescue missions, poses challenges due to volatile mission environments, such as weather conditions. While testing processes and methodologies are well established, structured guidance and execution support for field tests are still weak. This paper identifies requirements for field testing of drone missions and introduces the Field Testing Scenario Management (FiTS) approach for adaptive field testing guidance. FiTS aims to provide sufficient guidance for field testers as a foundation for efficient data collection to facilitate quality assurance and iterative improvement of field tests and CPSs. FiTS shall leverage concepts from scenario-based requirements engineering and Behavior-Driven Development to define structured and reusable test scenarios, with dedicated tasks and responsibilities for role-specific guidance. We evaluate FiTS by (i) applying it to three use cases for a search-and-rescue drone application to demonstrate feasibility and (ii) interviews with three experienced drone developers to assess its usefulness and collect further requirements. The study results indicate FiTS to be feasible and useful to facilitate drone field testing and data analysis.

Problem

• Drone Testing
• Reusable Scenarios
• Aerospace Rescue Missions

Innovation

• FiTS
• Drone Testing
• Real-Environment Validation