Tree-Based Deep Learning for Ranking Symbolic Integration Algorithms

2025-08-08

AI Summary

Symbolic integration algorithm selection traditionally relies on heuristic, expert-crafted rules, resulting in poor generalizability and low computational efficiency. To address this, we propose a two-stage deep learning framework grounded in expression trees: the first stage classifies applicable integration algorithms, while the second predicts output expression complexity and ranks candidate solutions. Our approach innovatively integrates Tree Transformers, learnable expression tree embeddings, and multi-generator synthetic data augmentation—effectively overcoming the limitations of sequence-based models in capturing hierarchical mathematical structure. Evaluated on a benchmark of 70,000 expressions, our model achieves 89.7% accuracy. Crucially, it demonstrates superior generalization on out-of-distribution test sets derived from Maple’s symbolic engine, significantly outperforming both Maple’s built-in selector and state-of-the-art machine learning baselines. These results validate the framework’s robustness, practicality, and potential to establish a new paradigm for automated decision-making in computer algebra systems.

Technology Category

• Machine Learning: Neuro-Symbolic Learning
• Cognitive Modeling & Cognitive Systems: Symbolic Representations
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Symbolic indefinite integration in Computer Algebra Systems such as Maple involves selecting the most effective algorithm from multiple available methods. Not all methods will succeed for a given problem, and when several do, the results, though mathematically equivalent, can differ greatly in presentation complexity. Traditionally, this choice has been made with minimal consideration of the problem instance, leading to inefficiencies. We present a machine learning (ML) approach using tree-based deep learning models within a two-stage architecture: first identifying applicable methods for a given instance, then ranking them by predicted output complexity. Furthermore, we find representing mathematical expressions as tree structures significantly improves performance over sequence-based representations, and our two-stage framework outperforms alternative ML formulations. Using a diverse dataset generated by six distinct data generators, our models achieve nearly 90% accuracy in selecting the optimal method on a 70,000 example holdout test set. On an independent out-of-distribution benchmark from Maple's internal test suite, our tree transformer model maintains strong generalisation, outperforming Maple's built-in selector and prior ML approaches. These results highlight the critical role of data representation and problem framing in ML for symbolic computation, and we expect our methodology to generalise effectively to similar optimisation problems in mathematical software.

Problem

• Selecting optimal symbolic integration algorithm efficiently
• Predicting output complexity of integration methods accurately
• Improving algorithm selection using tree-based deep learning

Innovation

• Tree-based deep learning for algorithm ranking
• Two-stage architecture for method selection
• Tree-structured expression representation boosts performance