finite-state transducers

Byte Pair Encoding (BPE) tokenization yields subword sequences that lack direct support for formal language operations, hindering rigorous pattern matching and compositional verification in open-vocabulary NLP systems. Method: We propose the first deterministic finite automaton (DFA) construction algorithm tailored to BPE output—treating tokenized sequences as constrained symbol strings without reconstructing original bytes or characters. Our approach introduces a novel equivalence-class partitioning scheme and transition function synthesis mechanism grounded in BPE merge rules, enabling linear-time O(n) DFA construction while preserving semantic fidelity. Contribution/Results: The resulting DFA supports efficient subword-level regular expression matching, lexicon equivalence checking, and formal language composition operations. It significantly improves both efficiency and composability of pattern recognition and formal verification in open-vocabulary NLP, establishing foundational automata infrastructure for verifiable, scalable, tokenization-aware language processing.

This work addresses the algebraic analysis of game programs by proposing a minimization method for deterministic finite-state transducers (FSTs) specified as input-word-to-output-symbol mapping pairs. Unlike conventional automata-based optimization paradigms, it introduces relational programming into transducer synthesis, grounded in relational algebra to ensure mathematically provable minimality. The algorithm synthesizes compact, deterministic, and formally verifiable FSTs from arbitrary input–output pair sets, with structures strictly compliant with proof assistants (e.g., Coq or Isabelle). Its core contribution is a declarative synthesis framework that eliminates reliance on state enumeration and heuristic optimization—longstanding bottlenecks in automaton synthesis—while guaranteeing provably minimal transducers. This establishes a novel pathway for algebraic modeling of program semantics, enabling rigorous, machine-checked reasoning about input–output behavior in interactive systems.

This paper addresses efficient regular expression parsing and submatch extraction by proposing a unified framework based on Tagged Deterministic Finite Automata (TDFA). Methodologically, it presents for the first time a complete, systematic treatment of TDFA construction, optimization, and ambiguity resolution—including POSIX and leftmost-longest semantics—via rigorous pseudocode and step-by-step examples; supports both precompilation and on-the-fly determinization; and incorporates production-grade optimizations such as state merging and deferred transitions. Contributions include: (1) the first fully implemented TDFA solution supporting multiple semantic policies, dual determinization modes, and end-to-end optimization; and (2) empirical validation via both the RE2C generator and a standalone Java library, demonstrating substantial speedups over conventional NFA/DFA approaches across multiple benchmark suites—achieving state-of-the-art runtime performance.

Traditional binary correctness verification fails to capture quantitative system behaviors. Method: We propose the first automated toolkit for quantitative automata supporting six classical semantics—Inf, Sup, LimInf, LimSup, LimInfAvg, and LimSupAvg—and systematically address core decision problems: emptiness, inclusion, equivalence, and safety/liveness verification. Our approach introduces weighted transition modeling and a generalized value-function framework, integrating symbolic decision procedures, optimization solvers, and automata transformation techniques to enable extremal-value computation, safety-liveness decomposition, and real-time monitoring. Contribution/Results: Experiments demonstrate efficiency on inclusion checking, constant-function recognition, and online monitoring tasks. We release the first open-source benchmark suite for quantitative automata analysis, establishing a scalable, modular, and unified infrastructure for quantitative system verification.

This paper investigates the computational complexity of bisimilarity checking for register automata (RA) and their extension—fresh-register automata (FRA)—over infinite alphabets. We consider RAs under various register valuation constraints (e.g., injectivity, freshness), and examine bisimilarity, language equivalence, and extensions with visibly pushdown storage. Method: We combine techniques from group theory and computational group theory, game semantics (with precise depth characterization of attacker strategies), symmetry reduction, and compact relation representations. Contributions: (i) Bisimilarity for injective RAs is PSPACE-complete (and NP-complete for deterministic RAs); freshness does not increase complexity. (ii) We refute Sakamoto’s conjecture that language equivalence for deterministic RAs is in NP, establishing instead its EXPTIME-completeness. (iii) Adding visibly pushdown storage renders bisimilarity undecidable. Altogether, we provide a complete complexity classification of bisimilarity for three register models: EXPTIME-complete, PSPACE-complete, and NP-complete.

This study addresses the minimization problem for streaming transducers, proposing a general existence criterion applicable to various models, including sequential transducers and string-to-string/tree transducers. By leveraging formal language theory, automata theory, algebraic structures, and term rewriting systems, the work establishes the first unified minimization theory for a broad class of streaming transducers. Furthermore, it develops an effective and computable minimization algorithm tailored to a variant of string-to-tree transducers whose outputs are incrementally constructed terms. The results not only characterize universal conditions under which minimization is feasible but also enable efficient optimization for concrete transducer models, thereby significantly advancing both the formal theory and practical applicability of streaming transducers.

This work addresses the challenge of efficiently and securely validating nested-structure inputs, such as SQL parameters with parentheses, by introducing a novel approach that integrates ultrametric spaces with automata theory. The method defines a distinguishing-word ultrametric over language equivalence classes to construct depth-bounded deterministic finite automata. Leveraging the contraction mapping property of guarded language operators, it employs Picard iteration to converge to a unique language fixed point, thereby guaranteeing correctness up to a bounded nesting depth. Notably, this is the first application of ultrametric spaces to automaton construction, enabling runtime validation with O(1) memory overhead. The resulting mechanism serves as a highly efficient pre-filter in web application firewalls, offering strong security assurances while substantially reducing both computational and memory costs.

This study addresses the hyper-minimization problem for well-typed deterministic register automata (DRA), which seeks to simultaneously minimize both the number of states and the number of registers. Building upon analogues of classical finite automaton concepts adapted to the DRA setting, the work establishes a theoretical foundation and proposes a specialized algorithm grounded in register-type analysis, equivalence relations, and refinement techniques. This is the first approach to achieve joint minimization of states and registers for well-typed DRAs, thereby proving the decidability of the hyper-minimization problem. The resulting automaton is rigorously shown to attain global optimality in both state count and register count among all well-typed DRAs recognizing the same language.

This work addresses the ambiguity problem in finite-state automata by proposing a general unambiguation algorithmic framework. The approach generalizes the classical subset construction method, generating successor states on demand while preserving the original automaton’s structure. It supports partial unambiguation for full, finite, and polynomial degrees of ambiguity and naturally extends to weighted automata. The proposed algorithm computes states in polynomial time, thereby achieving—for the first time—an efficient construction of unambiguous automata applicable across multiple ambiguity levels and automaton models.