🤖 AI Summary
This study investigates the impact of temporal characteristics—such as pauses and overlaps—in synthetic multi-speaker conversations on automatic speech recognition (ASR) performance. For the first time, conversational timing is explicitly modeled as a controllable training variable by parameterizing pause and overlap distributions using exponential tilting families. Diverse synthetic data are generated in a four-dimensional space through Latin hypercube sampling combined with multi-objective Bayesian optimization to train ASR systems. Results demonstrate that increased speech overlap significantly reduces cpWER, whereas longer and more variable inter-turn gaps degrade performance. Although Bayesian optimization yields only modest gains, the core contribution lies in uncovering the causal mechanisms linking temporal structure to ASR accuracy, underscoring the importance of task-oriented temporal diagnostics over mere corpus similarity in synthetic data design.
📝 Abstract
Synthetic multi-speaker conversations are widely used to train conversational automatic speech recognition (ASR) systems, but it remains unclear which timing properties make simulated data most useful. This paper studies conversational timing as a controllable training variable rather than merely as a corpus statistic to be reproduced. We parameterize pause and overlap timing distributions with an exponential-tilting family estimated from multiple conversational corpora, and then explore the resulting four-dimensional parameter space with Latin hypercube sampling and multi-objective Bayesian optimization. Each sampled timing configuration is used to generate simulated training conversations, train an ASR system, and evaluate concatenated-permutation word and character error rates (cpWER and cpCER) on a Hungarian dialogue corpus. The results show that downstream ASR behavior is explained more directly by induced timing statistics than by raw simulator coordinates or corpus proximity. In particular, higher overlap exposure is associated with lower cpWER, whereas longer and more variable gaps are associated with higher cpWER; cpCER follows the same trend, but with weaker statistical support. Bayesian optimization yields modest aggregate improvements, but its main value is analytical: it produces controlled timing interventions that reveal an overlap--gap trade-off in simulated conversational training data. These findings suggest that realistic simulation should be complemented by task-relevant diagnostics of overlap, gap, and timing-variability profiles.