Problem
Research questions and friction points this paper is trying to address.
Deep Learning Compiler
Random Test Case Generation
Hardware-specific Errors
Effective Random Test Generation for Deep Learning Compilers
🤖 AI Summary
Generating semantically valid and constraint-compliant random deep learning models for compiler testing remains challenging due to the complexity of syntactic and semantic constraints inherent in deep learning frameworks.
Method: This paper proposes Isra, the first approach to enable efficient, deterministic modeling and solving of deep learning model syntax and semantics via a domain-specific, backtracking-free constraint solver. Isra tightly integrates constraint solving with randomized testing to support end-to-end integration testing across mainstream compilers—including TVMLite, Glow, and SophGo.
Results: Empirical evaluation uncovered 24 previously unreported defects in TVM, Glow, and SophGo—significantly outperforming existing randomized testing methods in detection rate. The results demonstrate Isra’s effectiveness and practicality in enhancing deep learning compiler robustness.
📝 Abstract
Deep learning compilers help address the difficulties of deploying deep learning models on diverse types of hardware. Testing deep learning compilers is highly crucial, because they are impacting countless AI applications that use them for model optimization and deployment. To test deep learning compilers, random testing, the testing method popularly used for compiler testing practices, faces the challenge of generating semantically valid test inputs, i.e., deep learning models that satisfy the semantic model specifications (in short as semantic specifications). To tackle this challenge, in this paper, we propose a novel approach named Isra, including a domain-specific constraint solver that resolves the constraints from the semantic specifications without backtracking. We implement and apply our approach to three popular real-world deep learning compilers including TVM, Glow, and a commercial compiler named SophGo. The evaluation results show that Isra is more effective than the state-of-the-art approaches and the baseline approaches on constructing valid test inputs for compiler-bug detection, and Isra successfully finds 24 previously unknown bugs in released versions of the three compilers. These results indicate Isra's effectiveness and practical value.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Isra Method
Deep Learning Testing
Compiler Error Detection
🔎 Similar Papers
No similar papers found.
