Infinite Problem Generator: Verifiably Scaling Physics Reasoning Data with Agentic Workflows

2026-03-15

AI Summary

This work addresses the scarcity of high-quality, verifiable training data for complex physical reasoning by proposing an agent-based workflow for infinite problem generation. It introduces a novel “formulas-as-code” paradigm, encoding physics problem solutions as executable Python programs to guarantee mathematical consistency and enable automatic verification. The study further identifies a strong linear correlation between the number of formulas and code length, leveraging this insight to devise a proxy-free metric for problem difficulty. Based on this framework, the authors construct ClassicalMechanicsV1, a dataset comprising 1,335 high-fidelity classical mechanics problems that collectively span 102 unique formulas (averaging 3.05 formulas per problem). The complete generation pipeline and evaluation report are publicly released to support reproducibility and future research.

Technology Category

• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Web data generation and simulation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Training large language models for complex reasoning is bottlenecked by the scarcity of verifiable, high-quality data. In domains like physics, standard text augmentation often introduces hallucinations, while static benchmarks lack the reasoning traces required for fine-tuning. We introduce the Infinite Problem Generator (IPG), an agentic framework that synthesizes physics problems with guaranteed solvability through a Formula-as-Code paradigm. Unlike probabilistic text generation, IPG constructs solutions as executable Python programs, enforcing strict mathematical consistency. As a proof-of-concept, we release ClassicalMechanicsV1, a high-fidelity corpus of 1,335 classical mechanics problems expanded from 165 expert seeds. The corpus demonstrates high structural diversity, spanning 102 unique physical formulas with an average complexity of 3.05 formulas per problem. Furthermore, we identify a Complexity Blueprint, demonstrating a strong linear correlation ($R^2 \approx 0.95$) between formula count and verification code length. This relationship establishes code complexity as a precise, proxy-free metric for problem difficulty, enabling controllable curriculum generation. We release the full IPG pipeline, the ClassicalMechanicsV1 dataset, and our evaluation report to support reproducible research in reasoning-intensive domains.

Problem

• verifiable data
• physics reasoning
• data scarcity
• reasoning traces
• hallucination

Innovation

• Infinite Problem Generator
• Formula-as-Code
• agentic workflow
• verifiable reasoning data
• complexity blueprint