Neural Networks as Universal Finite-State Machines: A Constructive Deterministic Finite Automaton Theory

2025-05-16

AI Summary

This paper addresses the fundamental question of whether feedforward neural networks (FNNs) can strictly implement universal finite-state machines (FSMs), with a focus on exact simulation of deterministic finite automata (DFAs). We introduce *state-transition depth unfolding*: a method that maps DFA state transitions layer-by-layer onto the depth of an FNN, using ReLU or threshold activation functions to realize discrete symbolic computation. We provide the first constructive proof and explicit network architecture, establishing tight bounds on depth, width, and state compression. Our analysis reveals the linear separability of DFA transitions, the exponential state-compression capacity of threshold activations, and the embeddability of Myhill–Nerode equivalence classes as linearly separable regions in continuous latent space. We theoretically prove that fixed-depth FNNs cannot recognize non-regular languages—a result corroborated by both constructive examples and empirical validation. These findings establish a rigorous bridge between automata theory and neural-symbolic computation.

Technology Category

Machine Learning: Deep Neural Architectures and Foundation ModelsCognitive Modeling & Cognitive Systems: Neural Spike CodingKnowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphsSocial Networks and Social Media: Computational social scienceSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We present a complete theoretical and empirical framework establishing feedforward neural networks as universal finite-state machines (N-FSMs). Our results prove that finite-depth ReLU and threshold networks can exactly simulate deterministic finite automata (DFAs) by unrolling state transitions into depth-wise neural layers, with formal characterizations of required depth, width, and state compression. We demonstrate that DFA transitions are linearly separable, binary threshold activations allow exponential compression, and Myhill-Nerode equivalence classes can be embedded into continuous latent spaces while preserving separability. We also formalize the expressivity boundary: fixed-depth feedforward networks cannot recognize non-regular languages requiring unbounded memory. Unlike prior heuristic or probing-based studies, we provide constructive proofs and design explicit DFA-unrolled neural architectures that empirically validate every claim. Our results bridge deep learning, automata theory, and neural-symbolic computation, offering a rigorous blueprint for how discrete symbolic processes can be realized in continuous neural systems.

Problem

• Proving neural networks can simulate deterministic finite automata exactly
• Characterizing required depth and width for DFA simulation in neural networks
• Establishing expressivity boundaries of feedforward networks for non-regular languages

Innovation

• Unroll DFA transitions into neural layers
• Use binary threshold for state compression
• Embed Myhill-Nerode classes in latent spaces