Internalizing Geometric Law: Learning from Solver Residuals for Precision-Critical Generation

2026-06-08

AI Summary

This work addresses the hallucination problem in large language models when applied to precision-sensitive tasks such as technical drawing, where strict geometric constraints are often violated. To mitigate this, the authors propose PyGeoX, a programmable, geometry-specific domain language that compiles natural language descriptions into exact geometric constructions satisfying multiple constraints. By introducing a differentiable constraint loss and a saturation-based additive reward (SAR) mechanism that decomposes rewards per constraint, the approach effectively circumvents the issue of anomalous constraints masking gradients under global norm-based rewards, thereby preserving learning signals from partial progress. Experiments demonstrate that SAR improves solution success rates by 2.3× over an MSE-reward baseline on challenging tasks, and the resulting 8B-parameter model achieves performance on the newly introduced hierarchical benchmark PyGeoX-Bench that rivals substantially larger state-of-the-art systems.

Abstract

Large Language Models frequently hallucinate in precision-critical domains such as technical diagramming and mechanical design, where outputs must satisfy strict geometric constraints. We study open-ended geometric synthesis from natural language: translating free-form descriptions into precise constructions whose entities must simultaneously satisfy dozens of interacting constraints. To make this tractable, we release PyGeoX, a programmable geometric DSL that compiles declarative constraints into a differentiable loss, and PyGeoX-Bench, a stratified suite of 300 problems with per-constraint verifiable rewards. Using PyGeoX as a verifier, we identify a failure mode we call Outlier Gradient Masking: under global-norm rewards (any scheme that aggregates residuals through a single norm, for example, $\exp(-\mathrm{MSE})$), a single outlier constraint can nullify the learning signal across all others. To address this, we propose Saturating Additive Rewards (SAR), which decompose the reward into bounded per-constraint terms, preserving partial progress and ensuring consistent gradients even under severe violations. Against MSE-based rewards, the natural baseline for geometry solvers, SAR improves the hard-tier solving rate by $2.3\times$, and the resulting 8B model is competitive with much larger frontier systems on this benchmark. We release the engine, benchmark, and data at https://github.com/Huawei-AI4Math/PyGeoX.

Problem

• geometric constraints
• precision-critical generation
• large language models
• geometric synthesis
• solver residuals

Innovation

• Saturating Additive Rewards
• Geometric Constraint Satisfaction
• Differentiable DSL
• Outlier Gradient Masking
• Precision-Critical Generation