Compiler Optimization Testing Based on Optimization-Guided Equivalence Transformations

2025-04-06

AI Summary

Current compiler optimization testing heavily relies on multi-compiler differential testing, which is constrained by functional consistency requirements and incapable of detecting shared bugs or unimplemented features—leading to false positives and high manual verification overhead. To address this, we propose a single-compiler metamorphic testing approach driven by optimization rules: it formally models optimization preconditions, synthesizes input programs satisfying those conditions, injects semantics-preserving equivalent transformations, and automatically detects incorrect optimizations by comparing program outputs before and after transformation. This method breaks from the differential testing paradigm and introduces the first endogenous testing framework—“optimization-guided generation, equivalence-based mutation, output-driven validation”—integrated directly into GCC and LLVM’s low-level optimization pipelines. Evaluated on real-world scenarios, it uncovered five previously unreported optimization bugs, significantly enhancing the reliability assurance of compiler optimization implementations.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Optimization
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Compiler optimization techniques are inherently complex, and rigorous testing of compiler optimization implementation is critical. Recent years have witnessed the emergence of testing approaches for uncovering incorrect optimization bugs, but these approaches rely heavily on the differential testing mechanism, which requires comparing outputs across multiple compilers. This dependency gives rise to important limitations, including that (1) the tested functionality must be consistently implemented across all compilers and (2) shared bugs remain undetected. Thus, false alarms can be produced and significant manual efforts will be required. To overcome the limitations, we propose a metamorphic testing approach inspired by compiler optimizations. The approach is driven by how to maximize compiler optimization opportunities while effectively judging optimization correctness. Specifically, our approach first employs tailored code construction strategies to generate input programs that satisfy optimization conditions, and then applies various compiler optimization transformations to create semantically equivalent test programs. By comparing the outputs of pre- and post-transformation programs, this approach effectively identifies incorrect optimization bugs. We conducted a preliminary evaluation of this approach on GCC and LLVM, and we have successfully detected five incorrect optimization bugs at the time of writing. This result demonstrates the effectiveness and potential of our approach.

Problem

• Testing compiler optimizations without relying on multiple compilers
• Detecting shared bugs missed by differential testing approaches
• Ensuring correct optimization via metamorphic testing and equivalence checks

Innovation

• Metamorphic testing for compiler optimizations
• Tailored code construction for optimization conditions
• Semantic equivalence comparison for bug detection