Breaking the Data Barrier in Learning Symbolic Computation: A Case Study on Variable Ordering Suggestion for Cylindrical Algebraic Decomposition

2026-01-20

AI Summary

This work addresses the scarcity of labeled data in symbolic computation—caused by the high computational cost of exact high-dimensional operations—which hinders the application of supervised deep learning to tasks such as variable ordering in cylindrical algebraic decomposition (CAD). To overcome this limitation, we propose a Transformer-based pretraining-finetuning framework. The model is first pretrained on proxy tasks designed to efficiently generate synthetic data, then transferred and fine-tuned on real CAD variable ordering problems. This approach effectively alleviates the labeled data bottleneck and significantly outperforms the best existing heuristic strategies on public CAD benchmarks. To our knowledge, this is the first demonstration that large-scale pretraining can enhance performance on core symbolic computation tasks such as CAD variable ordering.

Technology Category

• Machine Learning: Neuro-Symbolic Learning
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Computer Vision: Visual Reasoning & Symbolic Representations

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Symbolic computation, powered by modern computer algebra systems, has important applications in mathematical reasoning through exact deep computations. The efficiency of symbolic computation is largely constrained by such deep computations in high dimension. This creates a fundamental barrier on labelled data acquisition if leveraging supervised deep learning to accelerate symbolic computation. Cylindrical algebraic decomposition (CAD) is a pillar symbolic computation method for reasoning with first-order logic formulas over reals with many applications in formal verification and automatic theorem proving. Variable orderings have a huge impact on its efficiency. Impeded by the difficulty to acquire abundant labelled data, existing learning-based approaches are only competitive with the best expert-based heuristics. In this work, we address this problem by designing a series of intimately connected tasks for which a large amount of annotated data can be easily obtained. We pre-train a Transformer model with these data and then fine-tune it on the datasets for CAD ordering. Experiments on publicly available CAD ordering datasets show that on average the orderings predicted by the new model are significantly better than those suggested by the best heuristic methods.

Problem

• symbolic computation
• cylindrical algebraic decomposition
• variable ordering
• data scarcity
• supervised learning

Innovation

• symbolic computation
• cylindrical algebraic decomposition
• variable ordering
• data-efficient learning
• Transformer pre-training