Efficient Construction of Large Search Spaces for Auto-Tuning

2025-09-30

AI Summary

In automated performance tuning, search-space construction—hampered by combinatorial explosion and intricate constraints—often takes minutes to days, constituting a critical bottleneck. Existing approaches (e.g., chain-of-trees) lack formal foundations and suffer from limited applicability. This paper introduces the first constraint-satisfaction-problem (CSP) formulation of search-space construction for auto-tuning. We propose a constraint-structure optimization strategy tailored to auto-tuning, integrating runtime parsing, declarative translation of user-specified constraints, and solver-aware co-optimization. Our approach is fully implemented in an open-source toolchain. Experiments demonstrate speedups of four orders of magnitude over brute-force enumeration, three orders over unoptimized CSP solvers, and one to two orders over state-of-the-art frameworks (e.g., TVM, Halide). Moreover, our method enables scalable, general-purpose tuning across larger and more complex configuration spaces.

Technology Category

• Constraint Satisfaction and Optimization: Search
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Auto ML and Hyperparameter Tuning

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Automatic performance tuning, or auto-tuning, accelerates high-performance codes by exploring vast spaces of code variants. However, due to the large number of possible combinations and complex constraints, constructing these search spaces can be a major bottleneck. Real-world applications have been encountered where the search space construction takes minutes to hours or even days. Current state-of-the-art techniques for search space construction, such as chain-of-trees, lack a formal foundation and only perform adequately on a specific subset of search spaces. We show that search space construction for constraint-based auto-tuning can be reformulated as a Constraint Satisfaction Problem (CSP). Building on this insight with a CSP solver, we develop a runtime parser that translates user-defined constraint functions into solver-optimal expressions, optimize the solver to exploit common structures in auto-tuning constraints, and integrate these and other advances in open-source tools. These contributions substantially improve performance and accessibility while preserving flexibility. We evaluate our approach using a diverse set of benchmarks, demonstrating that our optimized solver reduces construction time by four orders of magnitude versus brute-force enumeration, three orders of magnitude versus an unoptimized CSP solver, and one to two orders of magnitude versus leading auto-tuning frameworks built on chain-of-trees. We thus eliminate a critical scalability barrier for auto-tuning and provide a drop-in solution that enables the exploration of previously unattainable problem scales in auto-tuning and related domains.

Problem

• Automating construction of large search spaces for performance tuning
• Addressing slow search space generation in constraint-based auto-tuning
• Solving scalability bottlenecks in auto-tuning through CSP formulation

Innovation

• Reformulating auto-tuning as a Constraint Satisfaction Problem
• Developing a runtime parser for solver-optimal constraint translation
• Optimizing solver to exploit auto-tuning constraint structures