Evaluating Mutation-based Fault Localization for Quantum Programs

2025-05-14

AI Summary

This paper addresses the challenge of debugging quantum programs by conducting the first systematic evaluation of mutation-based fault localization (MBFL) on real-world quantum programs. We introduce quantum-specific mutation operators targeting qubit-level operations and implement spectrum-based fault localization (SFL) within the Qiskit framework, using the EXAM metric to quantify effectiveness. Our contributions are threefold: (1) we construct the first quantum MBFL benchmark comprising 23 real faults; (2) we empirically reveal that real faults are significantly harder to localize than synthetically seeded ones—a critical insight previously unreported; and (3) we demonstrate that MBFL achieves moderate effectiveness for shallow circuits (median EXAM = 19.4%), but its performance degrades substantially for highly entangled faults. These findings provide foundational empirical evidence and practical guidance for advancing automated debugging techniques in quantum software engineering.

Technology Category

• Machine Learning: Quantum Machine Learning
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Quantum computers leverage the principles of quantum mechanics to execute operations. They require quantum programs that define operations on quantum bits (qubits), the fundamental units of computation. Unlike traditional software development, the process of creating and debugging quantum programs requires specialized knowledge of quantum computation, making the development process more challenging. In this paper, we apply and evaluate mutation-based fault localization (MBFL) for quantum programs with the aim of enhancing debugging efficiency. We use quantum mutation operations, which are specifically designed for quantum programs, to identify faults. Our evaluation involves 23 real-world faults and 305 artificially induced faults in quantum programs developed with Qiskit(R). The results show that real-world faults are more challenging for MBFL than artificial faults. In fact, the median EXAM score, which represents the percentage of the code examined before locating the faulty statement (lower is better), is 1.2% for artificial benchmark and 19.4% for the real-world benchmark in the worst-case scenario. Our study highlights the potential and limitations of MBFL for quantum programs, considering different fault types and mutation operation types. Finally, we discuss future directions for improving MBFL in the context of quantum programming.

Problem

• Evaluating mutation-based fault localization for quantum programs
• Identifying faults in quantum programs using quantum mutation operations
• Assessing MBFL performance on real-world vs artificial quantum faults

Innovation

• Apply mutation-based fault localization to quantum programs
• Use quantum-specific mutation operations for fault identification
• Evaluate with real-world and artificial quantum program faults