StepAudio 3 Realtime Technical Report

2026-09-12

AI Summary

为实现实时口语互动，StepAudio 3 Realtime通过连续听-说-思-行循环模型，结合深度感知和无缝双工技术，并采用边说边思考的方法，有效解决了深入推理与低延迟之间的矛盾。

Abstract

Realtime spoken interaction demands deep reasoning, prompt responses, and fluid turn-taking. We present StepAudio 3 Realtime, an audio-language foundation model organized around a continuous listen-converse-think-act loop. Deep Perception captures rich acoustic cues to interpret user intent, while Seamless Duplex models synchronized audio streams to handle pauses, backchannels, and interruptions naturally. Crucially, we resolve the tension between deep deliberation and latency via Think-While-Speaking, executing private reasoning in parallel with spoken delivery. In reasoning mode, StepAudio 3 reaches a 73.0 macro average on StepAudioChat. With Think-While-Speaking, it achieves dialogue and reasoning performance comparable to dedicated reasoning models while speaking in real time. Furthermore, an integrated Voice Agent handles asynchronous tool execution without disrupting the dialogue flow. StepAudio 3 Realtime achieves top-tier performance across key dimensions: an exceptional 90.6 on the MMSU benchmark, 98.9 Overall on the Artificial Analysis Full-Duplex Bench, and a 56.0% macro task-success rate on $τ$-Voice.

Problem

• Realtime Spoken Interaction
• Deep Reasoning
• Latency
• Turn-taking
• Seamless Duplex

Innovation

• Continuous Listen-Converse-Think-Act Loop
• Deep Perception
• Seamless Duplex
• Think-While-Speaking
• Integrated Voice Agent