Symbolic Feedforward Networks for Probabilistic Finite Automata: Exact Simulation and Learnability

2025-09-12

AI Summary

This work addresses the problem of how symbolic neural networks can precisely model and differentiably learn probabilistic finite automata (PFAs). We propose a class of symbolic feedforward neural networks that represent state distributions as vectors and encode transitions as stochastic matrices, propagating probabilistic dynamics via matrix-vector multiplication—enabling acyclic, parallelizable, and differentiable computation. We establish, for the first time, a rigorous equivalence between PFAs and this network class, formally characterizing core automata-theoretic concepts—including probabilistic subset construction and ε-closure—thereby guaranteeing exact simulation and end-to-end gradient-based learning. The model eschews recurrent architectures, unifying symbolic computation with linear algebraic operations and introducing a soft update mechanism. Experiments demonstrate that standard gradient optimization on labeled sequences fully recovers the true PFA’s transition behavior, validating both expressive completeness and exact learnability. This provides a novel algebraic pathway toward unifying symbolic systems and deep learning.

Technology Category

Machine Learning: Neuro-Symbolic LearningCognitive Modeling & Cognitive Systems: Symbolic RepresentationsReasoning under Uncertainty: Relational Probabilistic Models

Application Category

Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingSemantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics

Abstract

We present a formal and constructive theory showing that probabilistic finite automata (PFAs) can be exactly simulated using symbolic feedforward neural networks. Our architecture represents state distributions as vectors and transitions as stochastic matrices, enabling probabilistic state propagation via matrix-vector products. This yields a parallel, interpretable, and differentiable simulation of PFA dynamics using soft updates-without recurrence. We formally characterize probabilistic subset construction, $varepsilon$-closure, and exact simulation via layered symbolic computation, and prove equivalence between PFAs and specific classes of neural networks. We further show that these symbolic simulators are not only expressive but learnable: trained with standard gradient descent-based optimization on labeled sequence data, they recover the exact behavior of ground-truth PFAs. This learnability, formalized in Proposition 5.1, is the crux of this work. Our results unify probabilistic automata theory with neural architectures under a rigorous algebraic framework, bridging the gap between symbolic computation and deep learning.

Problem

• Exact simulation of probabilistic finite automata using neural networks
• Learnability of symbolic simulators via gradient descent optimization
• Bridging probabilistic automata theory with neural architectures

Innovation

• Symbolic feedforward neural networks simulate PFAs
• Matrix-vector products enable probabilistic state propagation
• Gradient descent learns exact PFA behavior