Theorist Toolbox: Tools for Agent Based LLM-assisted economic theory Research

Unknown Date

AI Summary

This study addresses the lack of reusable tools in economic theory research and the unreliability of mathematical proofs generated by large language models due to hallucinations. To mitigate these issues, the authors propose a multi-agent collaborative framework centered on formal verification mechanisms. By integrating adversarial proof–verification pairs and a review-gated architecture, the framework anchors result reliability on external validation rather than the capabilities of any single model. Implemented using Claude Opus 4.8 and OpenAI Codex, the system successfully derives a consistent externality kernel in the Gans–Kominers grade inflation model and accurately identifies multiple erroneous claims. These results demonstrate the framework’s critical role in enhancing the rigor and trustworthiness of theoretical derivations in economics.

Abstract

Empirical economists often start their projects with a toolbox. Shared packages, replication archives, and circulated guides shorten the time between and idea and a rough initial draft. Theorists, on the other-hand, largely start from a blank page. By 2026, large language models can a produce and check nontrivial mathematics. The can also hallucinate and write wrong claims very convincingly. The current bottleneck on machine-assisted theory is no longer production but trust: a model will claim to prove a false theorem as readily as a true one. Building on recent attempts in mathematics, I present 3 methods for doing economic theory with a language model. These methods differ on how the work is verified: a single disciplined pass, an adversarial prover-verifier pair (Claude Opus~4.8 proposing, OpenAI Codex refuting), and a structured multi-agent project with a reviewer gate (inspired by the Google co-mathematician architecture). I demonstrate these protocols on one open worked example: designing a Groves/Pigouvian incentive mechanism for the Gans--Kominers eigengrade model of grade inflation. None of the three runs produced a strict direct-revelation VCG/Clarke mechanism (as requested, perhaps due to the non-existence of such mechanism). Three phenomena recur. First, convergent discovery: two runs derive the same effective-resistance externality kernel on opposite margins. Second, adversarial verification is load-bearing: the pair caught three of its own false claims and the gate rejected a sub-goal. Third, polish is not rigor: the most finished-looking output was the least verified. The methodological takeaway is that external verification, not model capability, is the design variable.

Problem

• trust
• verification
• large language models
• economic theory
• hallucination

Innovation

• agent-based LLM
• adversarial verification
• economic theory
• mechanism design
• external verification