🤖 AI Summary
This work systematically translates classical impossibility results—such as Turing undecidability, Arrow’s impossibility theorem, and the no-free-lunch theorem—into quantifiable design principles for trustworthy AI. It introduces the notion of a “deterministic horizon,” an accuracy upper bound computable a priori solely from model depth and embedding width. By integrating information theory, circuit complexity, and residual stream analysis, the study reveals fundamental capability limits in multi-stage retrieval and preference learning. Empirical evaluation across twelve Transformer architectures shows horizons between 19–31 layers, with optimal fine-tuning recovering less than 4% accuracy. Furthermore, quantified neural inference incurs zero-knowledge verification overheads of 110–190× per nonlinear activation. The paper culminates in a framework comprising sixteen paired specifications, two of which have already been formally proven in combination.
📝 Abstract
Large language models now write software, draft legal documents, and produce clinical notes, yet fundamental limits, from Turing and Arrow to the No Free Lunch theorems, shape what computation can do. This thesis turns such impossibility results from curiosities into design rules. Its flagship result proves an accuracy ceiling set by architecture alone: past a critical reasoning depth, no amount of training moves it, at any adapter rank, sample size, or loss function. Computable before deployment from layer count and embedding width, this Deterministic Horizon is measured between nineteen and thirty-one across twelve transformer architectures, and fine-tuning on optimal-length traces recovers under four percentage points. The mechanism is a capacity invariant of the residual stream, and an information-theoretic conversion yields super-exponential accuracy decay past the horizon. An unconditional circuit-complexity lower bound for modular exponentiation against constant-depth prime-modulus circuits complements this result. The same argument recasts across subfields: preference learning under any misspecified model jumps discontinuously in sample complexity; multi-stage retrieval pipelines require at least as many independent metrics as stages; standard truthful auctions fail for agents with prompt-dependent valuations; and zero-knowledge verification of neural inference pays a measured overhead of one hundred ten to one hundred ninety times per non-linear activation. Together these form a catalogue of sixteen specifications, each pairing a computable boundary, a quantified violation cost, and a constructive design rule: two compositions are proved, one pairing is an honest obstruction, and four remain open. The impossibility-specification methodology is offered for the generative research programme that trustworthy AI may need. Every fundamental limit of AI is also a design rule.