BeaverTalk: Oregon State University's IWSLT 2025 Simultaneous Speech Translation System

2025-05-29

AI Summary

This paper addresses the IWSLT 2025 simultaneous speech translation task (English→German/Chinese) by proposing BeaverTalk, an end-to-end cascaded real-time system supporting both high- and low-latency modes. Methodologically, it employs VAD-based streaming segmentation and Whisper Large V2 for ASR, coupled with LoRA-efficient fine-tuning of Gemma-3-12B for streaming translation; it further introduces a novel source-language sentence-level memory bank–driven conversational prompting strategy to significantly improve contextual coherence and translation quality under low latency. Experiments show that BeaverTalk achieves state-of-the-art BLEU scores of 27.83 (En→De) and 37.23 (En→Zh) in high-latency settings—topping the official IWSLT 2025 benchmark. Key contributions include: (1) a lightweight, efficient large language model streaming adaptation framework; (2) a source-language memory–enhanced prompting mechanism tailored for low-latency translation; and (3) a fully open-sourced, reproducible end-to-end simultaneous translation system.

Technology Category

Machine Learning: Large Multimodal Models (LMMs)Natural Language Processing: Language Grounding & Multi-modal NLPData Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web searchUser Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systemsSystems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications

Abstract

This paper discusses the construction, fine-tuning, and deployment of BeaverTalk, a cascaded system for speech-to-text translation as part of the IWSLT 2025 simultaneous translation task. The system architecture employs a VAD segmenter for breaking a speech stream into segments, Whisper Large V2 for automatic speech recognition (ASR), and Gemma 3 12B for simultaneous translation. Regarding the simultaneous translation LLM, it is fine-tuned via low-rank adaptors (LoRAs) for a conversational prompting strategy that leverages a single prior-sentence memory bank from the source language as context. The cascaded system participated in the English$ ightarrow$German and English$ ightarrow$Chinese language directions for both the low and high latency regimes. In particular, on the English$ ightarrow$German task, the system achieves a BLEU of 24.64 and 27.83 at a StreamLAAL of 1837.86 and 3343.73, respectively. Then, on the English$ ightarrow$Chinese task, the system achieves a BLEU of 34.07 and 37.23 at a StreamLAAL of 2216.99 and 3521.35, respectively.

Problem

• Building a cascaded system for simultaneous speech-to-text translation
• Fine-tuning a simultaneous translation LLM using LoRAs and context memory
• Evaluating performance on English→German and English→Chinese tasks

Innovation

• VAD segmenter for speech stream segmentation
• Whisper Large V2 for ASR
• Gemma 3 12B with LoRAs for translation