🤖 AI Summary
This work investigates the geometric reasoning capabilities of Graph Neural Networks (GNNs) and Transformers in embedding space, focusing on reconstructing implicit 2D geometric structures—specifically, predicting spatial coordinates and recovering underlying shapes from point sets defined by discrete geometric constraints on a 2D grid.
Method: We propose a geometry-aware GNN architecture explicitly designed for geometric reasoning. Crucially, it operates without explicit coordinate supervision, relying solely on relational graph structure.
Contribution/Results: We demonstrate, for the first time, that the learned node embeddings spontaneously organize into a low-dimensional subspace preserving neighborhood relationships—effectively recovering the latent 2D grid topology. Quantitatively, our GNN significantly outperforms Transformer baselines in both prediction accuracy and scalability. Qualitative analysis confirms that the embedding space faithfully encodes geometric structure, providing strong evidence of implicit geometric modeling capacity. These findings establish a novel, interpretable paradigm for spatial reasoning grounded in learned embeddings.
📝 Abstract
In this contribution, we demonstrate that Graph Neural Networks and Transformers can learn to reason about geometric constraints. We train them to predict spatial position of points in a discrete 2D grid from a set of constraints that uniquely describe hidden figures containing these points. Both models are able to predict the position of points and interestingly, they form the hidden figures described by the input constraints in the embedding space during the reasoning process. Our analysis shows that both models recover the grid structure during training so that the embeddings corresponding to the points within the grid organize themselves in a 2D subspace and reflect the neighborhood structure of the grid. We also show that the Graph Neural Network we design for the task performs significantly better than the Transformer and is also easier to scale.