Low-Latency Turn-Taking via Context-Aware Preface Generation in a Real-World Dialogue Robot

2026-07-25

AI Summary

Current LLM-driven dialogue systems typically wait for speech recognition to complete before generating a response, resulting in noticeable latency. While fixed filler phrases can mitigate this delay, they often lack naturalness. This work proposes a two-stage incremental framework that triggers the generation of context-aware lead-in utterances as soon as user intent becomes predictable, and leverages a Voice Activity Prediction (VAP) model to dynamically determine the optimal timing for their delivery. By decoupling intent-readiness detection from context-sensitive lead-in generation—a first in this domain—the approach effectively balances low latency with conversational fluency. Experimental results show that although context-aware lead-ins introduce a slight increase in initial latency, they significantly reduce the interval between the lead-in and the main response, thereby enhancing the overall naturalness of dialogue pacing.

Abstract

Large language model (LLM)-based dialogue systems suffer response delays because generation begins only after final speech recognition. While fixed fillers are a workaround, they become unnatural over time. We propose a two-stage incremental framework that decouples prefatory-response preparation from speech onset. Once user intent becomes predictable, an intent readiness detector triggers LLM-based generation of a short prefatory response. Concurrently, a voice activity projection (VAP) model determines when to deliver it. Through a field experiment with a route-guidance robot in a shopping mall, we evaluated three conditions: no-filler, fixed-filler, and contextual-preface. Both fixed-filler and contextual-preface significantly reduced initial response latency relative to no-filler. Relative to fixed-filler, contextual-preface had significantly longer initial response latency but a significantly shorter initial-to-main gap. Exploratory ratings showed no significant differences. These results indicate a timing trade-off.

Problem

• low-latency
• turn-taking
• dialogue systems
• response delay
• context-aware

Innovation

• context-aware prefacing
• incremental dialogue generation
• intent readiness detection
• voice activity projection
• low-latency turn-taking