Can AI Refute Economic Theory? Evidence from Beyond the Knowledge Cutoff

2026-06-03

AI Summary

This study presents the first systematic evaluation of whether large language models (LLMs)—including ChatGPT, Claude, and Gemini—can independently identify and correct errors in economic theory. By prompting multiple state-of-the-art models to scrutinize four theoretical papers containing known flaws, the authors assess their capacity to detect logical inconsistencies and generate counterexamples, employing both prompt engineering and minimal human guidance. Results indicate that while ChatGPT Pro occasionally produces valid counterexamples or corrected proofs, no model reliably pinpoints the actual errors without substantial human intervention. The work proposes a novel human–AI collaborative framework that may surpass conventional peer review in efficacy and lays a methodological foundation for AI-assisted theoretical validation.

Abstract

Can artificial intelligence (AI) refute economic theory? I document experiments in which I asked several AI models (Gemini, Refine, Claude, and ChatGPT) to check the correctness of four published papers in economic theory, each containing an error that I helped identify or correct. ChatGPT Pro performed best, occasionally constructing counterexamples and corrected proofs, while other models fared worse. However, no model located a true error without substantial human guidance, and data contamination complicates interpretation. I argue that a competent human paired with a frontier model can outperform current peer review, but AI cannot yet refute economic theory on its own.

Problem

• artificial intelligence
• economic theory
• refutation
• peer review
• AI evaluation

Innovation

• AI-assisted peer review
• economic theory validation
• large language models
• counterexample generation
• human-AI collaboration