Thermodynamic Signatures of Reasoning: Free-Energy and Spectral-Form-Factor Diagnostics for Hallucination Detection in Large Language Models

2026-06-17

AI Summary

Detecting hallucinations in large language models urgently requires efficient, fine-tuning-free methods. This work proposes Free-Energy Signatures (FES), which uniquely integrate thermodynamic potentials—such as free energy and spectral entropy—with random matrix theory (RMT). By treating the attention map Laplacian as a Hamiltonian, FES constructs unsupervised spectral descriptors that exhibit Lipschitz stability, strong expressive power, and theoretical Probably Approximately Correct (PAC) guarantees. Empirical evaluation across six open-source large language models and six benchmark datasets demonstrates that FES achieves an average AUROC 6.5 points higher than LapEig and 2.4 points higher than GoR-4, with an RMT-based anomaly score reaching 0.71 AUROC.

Abstract

Hallucination detection in large language models (LLMs) is deployment-critical, and recent work shows that the spectrum of attention-derived graph Laplacians carries strong signal about reasoning quality. Prior spectral diagnostics, however, summarize the Laplacian spectrum by a handful of eigenvalues or hand-picked scalars, leaving most of its structure unused. We propose Free-Energy Signatures (Fes), a spectral descriptor that treats each layer's attention Laplacian as a Hamiltonian and extracts its thermodynamic potentials partition function, free energy, spectral entropy, heat capacity together with the random-matrix-theory (RMT) spectral form factor. We prove three results: (i)~Lipschitz stability of Fes under attention perturbation; (ii)~an expressiveness result showing that Fes enriches finite spectral summaries and approximates moment-derived spectral functionals under explicit regularity and grid-resolution assumptions; and (iii)~a finite-sample PAC bound on the AUROC of a training-free detector built from Fes. Empirically, across six open-weight LLMs and six benchmarks, a lightweight probe on Fes descriptors achieves the strongest aggregate AUROC among attention-spectral baselines, improving over LapEig by $+6.5$ AUROC points and over GoR-4 by $+2.4$ points on average, while requiring no update to the underlying LLM. In the fully unsupervised setting, an RMT-deviation score achieves mean AUROC $0.71$, providing a label-free but weaker detector. A complementary RMT analysis shows that correct generations exhibit more Wigner-Dyson like spectral statistics, whereas hallucinations exhibit more Poisson-like statistics. The anonymized code and config are provided in the supplementary material.

Problem

• hallucination detection
• large language models
• spectral diagnostics
• attention mechanism
• reasoning quality

Innovation

• Free-Energy Signatures
• spectral form factor
• hallucination detection
• graph Laplacian
• random matrix theory