Delta Debugging for Cyber-Physical Systems with Flaky Test Executions

2026-07-28

AI Summary

This study addresses the challenge of reproducing failures in cyber-physical system (CPS) simulation testing, where non-deterministic behaviors often hinder consistent fault manifestation. To tackle this issue, the work extends delta debugging to stochastic CPS scenarios for the first time, introducing three novel delta debugging algorithms tailored for randomized environments. The proposed approach integrates statistical failure analysis, repeated execution, and environment-aware input minimization to identify a minimal triggering input while preserving fault semantics. Empirical evaluation on case studies involving elevator scheduling and autonomous mobile robots demonstrates that the method significantly enhances the stability of failure reproduction and substantially reduces debugging time, effectively mitigating execution flakiness inherent in such systems.

Abstract

Simulation-based testing is widely used to validate Cyber-Physical Systems (CPSs), yet modern CPS simulators frequently exhibit non-deterministic (flaky) behavior, making failures difficult to reproduce and debug. Although delta debugging has proven effective for deterministic systems, its underlying assumptions do not hold in stochastic environments. This paper presents three delta debugging algorithms that combine statistical failure analysis, repeated executions, and environment-aware reduction to isolate minimal failure-inducing test inputs for stochastic CPSs. We evaluate the proposed techniques on two complementary case study systems: an industrial elevator dispatching system employing stochastic optimization and an autonomous mobile robot exhibiting simulator-induced non-determinism. The results show that the proposed approaches substantially reduce debugging time while preserving the original failure behavior. More importantly, we observe that minimizing failure-inducing test inputs frequently increases failure reproducibility compared with the original executions. By eliminating execution segments that introduce incidental stochastic effects, the reduced test inputs isolate the causal conditions of the failure and consistently reproduce it with higher probability. These findings suggest that delta debugging not only simplifies failure analysis but also mitigates execution flakiness, providing a practical foundation for debugging CPSs.

Problem

• Cyber-Physical Systems
• flaky test executions
• delta debugging
• non-determinism
• failure reproducibility

Innovation

• delta debugging
• stochastic CPS
• flaky test
• failure reproducibility
• environment-aware reduction