Better Safe Than Sorry: Enhancing Arbitration Graphs for Safe and Robust Autonomous Decision-Making

2024-11-15

arXiv.org

AI Summary

To address safety risks arising from component failures in autonomous systems operating within complex, dynamic environments, this paper proposes an enhanced arbitration graph framework integrating formal verification with a hierarchical fallback mechanism. The method embeds a runtime safety verification module within the decision pipeline to ensure only formally verified safe commands are executed; introduces a structured fallback layer enabling graceful degradation to pre-verified safe behaviors upon component failure; and implements a header-only C++ library that supports incremental integration of experimental components while rigorously specifying their safety-enabling conditions. Evaluated in Pac-Man simulation and autonomous driving scenarios, the approach significantly reduces incident rates and demonstrates sustained safe operation under component anomalies.

Technology Category

• Natural Language Processing: Safety and Robustness
• Multiagent Systems: Adversarial Agents
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

This paper introduces an extension to the arbitration graph framework designed to enhance the safety and robustness of autonomous systems in complex, dynamic environments. Building on the flexibility and scalability of arbitration graphs, the proposed method incorporates a verification step and structured fallback layers in the decision-making process. This ensures that only verified and safe commands are executed while enabling graceful degradation in the presence of unexpected faults or bugs. The approach is demonstrated using a Pac-Man simulation and further validated in the context of autonomous driving, where it shows significant reductions in accident risk and improvements in overall system safety. The bottom-up design of arbitration graphs allows for an incremental integration of new behavior components. The extension presented in this work enables the integration of experimental or immature behavior components while maintaining system safety by clearly and precisely defining the conditions under which behaviors are considered safe. The proposed method is implemented as a ready to use header-only C++ library, published under the MIT License. Together with the Pac-Man demo, it is available at github.com/KIT-MRT/arbitration_graphs.

Problem

• Enhance safety in autonomous systems.
• Improve robustness in dynamic environments.
• Integrate experimental components safely.

Innovation

• Enhanced arbitration graph framework
• Verification step and fallback layers
• Header-only C++ library implementation