On the Turing Completeness of Transformers and Agents

2026-09-17

AI Summary

本文探讨了单个Transformer在处理无限长度输入时的推理局限性，并提出通过构建包含决策、执行和记忆模块的代理系统来实现图灵完备。

Abstract

Transformers have emerged as the dominant architecture in sequence modeling, achieving remarkable success in natural language processing and reasoning tasks. While existing literature has established the Turing completeness of transformers under bounded input length, the reasoning power of a single transformer operating on inputs of unbounded length is not fully explored. In this paper, we theoretically investigate the reasoning limitations of a single transformer and the enhanced capabilities of agent systems. We show that a single fixed finite precision transformer cannot memorize certain Turing machines with inputs of arbitrary length, such as the arithmetic; and a single fixed infinite precision transformer trained with a random algorithm is not Turing complete with probability one under reasonable conditions. To overcome the limitation of a single transformer, we define a formal agent architecture consisting of decision, execution, and memory modules and show that for any Turing machine $\mathbb{T}$, there exists an agent that can memorize $\mathbb{T}$ and is computationally the same as $\mathbb{T}$. Thus, agents are Turing complete.

Problem

• Turing completeness
• transformers
• unbounded length
• agents

Innovation

• Turing Completeness
• Transformers
• Agents
• Unbounded Input Length
• Agent Architecture