The Dynamic Articulatory Model DYNARTmo: Dynamic Movement Generation and Speech Gestures

2025-11-11

AI Summary

This study addresses the hierarchical mapping problem from abstract linguistic representations to concrete articulatory–acoustic realizations in speech production. To this end, we propose DYNARTmo—a dynamic articulatory model grounded in a neurobiologically inspired framework. DYNARTmo employs speech gestures and their score-like temporal representations (“gesture scores”) as control signals, driving a vocal-tract model via gesture coordination mechanisms and continuous trajectory generation to yield physiologically plausible articulator movements. It realizes an end-to-end, hierarchical computational simulation: language → gestures → articulation → acoustics. Key contributions include: (1) the first integration of gesture scores into dynamic articulatory modeling, explicitly encoding temporal coordination; (2) real-time, continuous articulatory synthesis for complex phonetic sequences; and (3) differentiable, biophysically constrained joint simulation of acoustic and articulatory parameters. Experiments demonstrate high-fidelity modeling of multisyllabic words and connected speech.

Technology Category

Cognitive Modeling & Cognitive Systems: Simulating Human BehaviorNatural Language Processing: SpeechKnowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systemsSemantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactionsGraph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

This paper describes the current implementation of the dynamic articulatory model DYNARTmo, which generates continuous articulator movements based on the concept of speech gestures and a corresponding gesture score. The model provides a neurobiologically inspired computational framework for simulating the hierarchical control of speech production from linguistic representation to articulatory-acoustic realization. We present the structure of the gesture inventory, the coordination of gestures in the gesture score, and their translation into continuous articulator trajectories controlling the DYNARTmo vocal tract model.

Problem

• Generating continuous articulator movements from speech gestures
• Simulating hierarchical control of speech production computationally
• Translating gesture scores into vocal tract articulatory trajectories

Innovation

• Generates continuous articulator movements from gestures
• Provides neurobiologically inspired hierarchical speech control
• Translates gesture scores into vocal tract trajectories