Learning to Coordinate Symbolic Tools: LLM Agents for Verified Sum-of-Squares Certificates

2026-07-31

AI Summary

This work addresses the challenge of enabling large language models to effectively invoke symbolic computation tools and orchestrate multiple non-unique algebraic transformations to produce verifiable weighted sum-of-squares (SOS) decompositions. To this end, the authors propose an agent framework that integrates algebraic task-specific supervised fine-tuning, symbolic tool invocation, and verifier feedback. The approach uniquely combines multi-task synthetic data training, supervised fine-tuning without native tool-use capabilities, and a novel Group Relative Policy Optimization (GRPO) algorithm guided by symbolic rewards. Evaluated on the weighted SOS task, the method achieves a verification success rate of 78.96%, substantially outperforming the baseline of 44.73%, and attains a macro accuracy of 91.75% across nine polynomial manipulation tasks, significantly enhancing the model’s verifiability and generalization in symbolic mathematical reasoning.

Abstract

Tool calling allows large language models (LLMs) to invoke external computation during problem solving, a useful capability in various fields including AI for mathematics. We study this setting through weighted sum-of-squares (SOS) decomposition, a machine-checkable route to proving polynomial nonnegativity and hence polynomial inequalities. A candidate decomposition can be checked exactly, but finding one requires choosing among non-unique regroupings and coordinating multiple symbolic transformations. We develop an agent that combines algebraic task training, symbolic tools, and verifier-grounded optimization for this task. Rather than training only on the composite SOS task, we construct 1.35 million synthetic examples covering eight supporting polynomial tasks together with weighted-SOS decomposition. We first apply supervised fine-tuning (SFT) to direct algebra problems and simulated symbolic traces, and then use Group Relative Policy Optimization (GRPO) with task-specific symbolic rewards. The SFT corpus contains no native tool-calling messages; at evaluation, the agent uses native SymPy calls for expansion, collection, reordering, and factorization. Every final SOS answer is checked by exact expansion and coefficient comparison. On held-out, same-generator synthetic problems, the full SFT+GRPO+tools system is the strongest of four evaluated configurations, reaching 78.96% verified success on weighted SOS, compared with 44.73% for the base model with the same tools, and 91.75% macro accuracy across nine polynomial tasks. Within this controlled setting, our work provides a case study of combining domain-specific skill training, executable tools, and verifier feedback, and may inform the design of tool-calling agents in other domains with exactly checkable outputs.

Problem

• sum-of-squares
• symbolic tools
• polynomial nonnegativity
• tool calling
• verified certificates

Innovation

• tool calling
• sum-of-squares decomposition
• symbolic reasoning
• verifier-grounded optimization
• large language model agents