Overcoming State Inertia in Full-Duplex Spoken Language Models via Activation Steering

2026-06-09

AI Summary

This study addresses a critical limitation in full-duplex spoken language models (FD-SLMs), which often fail to comprehend user utterances interrupted mid-speech due to delayed internal state transitions that cause loss of initial input content. The work formally defines and quantifies the previously uncharacterized “state inertia” phenomenon, uncovering the latent mechanism governing dynamic modulation between generative and perceptual states. To mitigate this issue, the authors propose a training-free activation-guided intervention strategy that leverages perceptual vector steering to accelerate state switching. Evaluated through hidden-state analysis and a zero-buffer benchmark (ZBB), the approach substantially improves interruption handling across multiple state-of-the-art FD-SLMs: for instance, PersonaPlex achieves a rise in interruption comprehension accuracy from 28% to 45% and an increase in initial-word capture rate from 40% to 72%.

Abstract

Full-duplex spoken language models (FD-SLMs) enable seamless speech interaction by allowing models to listen and speak simultaneously, yet the internal mechanism by which they coordinate listening and speaking remains underexplored. We analyze the predictive behavior encoded in FD-SLM hidden representations and find that they exhibit stream-specific predictive patterns: during listening, they preferentially predict the incoming user stream, whereas during speaking, they preferentially predict the model output stream. Building on this observation, we show that FD-SLMs dynamically modulate their internal predictive focus between two states: a generative state aligned with model output generation and a perceptive state aligned with incoming user input. However, this modulation can lag behind abrupt changes in conversational context. During user interruptions, the model remains transiently biased toward the generative state before transitioning into the perceptive state, causing it to miss the beginning of the incoming input. We term this delayed internal transition state inertia. To quantify its downstream impact, we introduce the Zero-Buffer Benchmark (ZBB), a diagnostic benchmark for evaluating immediate interruption comprehension when user speech begins abruptly. We evaluate this setting using response correctness and initial-word occurrence rate (IWOR). Finally, we mitigate state inertia through activation steering with a perception vector, a training-free intervention with little additional computational overhead. Across multiple state-of-the-art FD-SLMs, activation steering substantially improves interruption handling; for example, on PersonaPlex, it improves correctness from 28% to 45% and IWOR from 40% to 72% without any fine-tuning.

Problem

• state inertia
• full-duplex spoken language models
• interruption handling
• predictive focus
• conversational context

Innovation

• activation steering
• state inertia
• full-duplex spoken language models
• Zero-Buffer Benchmark
• perception vector