Neural Networks as Universal Finite-State Machines: A Constructive ReLU Simulation Framework for NFAs

2025-05-30

AI Summary

This work addresses the fundamental question of whether neural networks can exactly simulate nondeterministic finite automata (NFAs). We propose a constructive method: encoding NFA states as binary vectors, implementing transitions via sparse linear layers, and rigorously modeling nondeterministic branching, subset construction, and ε-closure using ReLU activations—without recurrence, external memory, or approximation. To our knowledge, this is the first symbol-level, formally provable simulation of NFAs in standard feedforward ReLU networks. We prove theoretically that any regular language recognized by an n-state NFA can be exactly recognized by a three-layer ReLU network with width O(n) per layer, and that gradient descent training preserves symbolic semantics. Experiments confirm 100% automaton alignment across path tracing, subset construction, ε-closure convergence, and acceptance decisions—demonstrating both theoretical soundness and empirical consistency.

Technology Category

Machine Learning: Neuro-Symbolic LearningNatural Language Processing: Sentence-level Semantics, Textual Inference, etc.Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphsSearch and Retrieval-Augmented AI: Large language models for searchSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We present a formal and constructive framework establishing the equivalence between nondeterministic finite automata (NFAs) and standard feedforward ReLU neural networks. By encoding automaton states as binary vectors and transitions as sparse linear layers, we show that ReLU activations simulate nondeterministic branching, subset construction, and $epsilon$-closures in a mathematically precise manner. Our core theoretical results prove that a three-layer ReLU network of width $mathcal{O}(n)$ can exactly recognize any regular language accepted by an $n$-state NFA-without recurrence, memory, or approximation. Furthermore, we show that gradient descent over structure-preserving networks preserves symbolic semantics and acceptance behavior. Extensive experiments across multiple validation tasks-including parallel path tracking, symbolic subset construction, $epsilon$-closure convergence, acceptance classification, structural training invariants, and functional equivalence-achieve perfect or near-perfect empirical alignment with ground-truth automata. This work provides the first provably complete symbolic simulation of NFAs within standard deep learning architectures, uniting automata theory with neural computation through ReLU dynamics.

Problem

• Simulating NFAs with ReLU neural networks
• Proving equivalence between NFAs and ReLU networks
• Ensuring exact regular language recognition

Innovation

• ReLU networks simulate NFAs precisely
• Three-layer ReLU network recognizes regular languages
• Gradient descent preserves symbolic semantics