HATSolver: Learning Groebner Bases with Hierarchical Attention Transformers

2025-12-09

AI Summary

This work addresses the symbolic learnability and scalability bottlenecks in Gröbner basis computation for solving systems of multivariate polynomial equations. We propose a symbolic-neural hybrid method based on a Hierarchical Attention Transformer (HAT). Our key contributions are threefold: (i) we introduce, for the first time, tree-structured inductive bias into the attention mechanism, enabling faithful modeling of polynomial ideals of arbitrary depth; (ii) we develop a theory-guided computational cost analysis framework, integrated with curriculum learning to overcome size limitations; and (iii) extensive experiments demonstrate that our approach significantly outperforms Kera et al. (2024) across multiple benchmarks—scaling problem size by an order of magnitude, reducing computational overhead by over 40%, and exhibiting superior robustness and generalization.

Technology Category

Machine Learning: Neuro-Symbolic LearningSearch and Optimization: Learning to SearchConstraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingEconomics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

At NeurIPS 2024, Kera et al. introduced the use of transformers for computing Groebner bases, a central object in computer algebra with numerous practical applications. In this paper, we improve this approach by applying Hierarchical Attention Transformers (HATs) to solve systems of multivariate polynomial equations via Groebner bases computation. The HAT architecture incorporates a tree-structured inductive bias that enables the modeling of hierarchical relationships present in the data and thus achieves significant computational savings compared to conventional flat attention models. We generalize to arbitrary depths and include a detailed computational cost analysis. Combined with curriculum learning, our method solves instances that are much larger than those in Kera et al. (2024 Learning to compute Groebner bases)

Problem

• Improves Groebner bases computation using Hierarchical Attention Transformers
• Models hierarchical relationships for computational efficiency in polynomial systems
• Solves larger polynomial equation instances via curriculum learning

Innovation

• Hierarchical Attention Transformers model tree-structured data
• Curriculum learning enables solving larger polynomial systems
• Generalizes to arbitrary depths with computational cost analysis