Problem
Research questions and friction points this paper is trying to address.
privacy-preserving
offline voice dictation
cross-platform
on-device speech recognition
hold-to-talk
YazSes: An Offline, Privacy-First, Cross-Platform Hold-to-Talk Voice-Dictation System
🤖 AI Summary
This work addresses the privacy risks inherent in cloud-based speech transcription services and the limited portability or usability of existing local alternatives by proposing a fully local, open-source push-to-talk transcription system. The system supports all three major desktop platforms through a unified codebase, leveraging protocol abstraction and a JSON-RPC control plane to ensure cross-platform compatibility. Built upon faster-whisper (CPU int8) for on-device speech recognition, it integrates regular-expression-based command parsing with optional small language model routing, guaranteeing telemetry-free operation and end-to-end encryption while delivering high accuracy and efficiency. Evaluated on LibriSpeech, the small.en model achieves a word error rate of 2.59% with a real-time factor of 0.520; command recognition attains 100% accuracy and 0% false-positive rate, with non-decoding overhead as low as 0.289 ms.
📝 Abstract
Cloud voice-dictation services deliver strong accuracy but require streaming a user's speech to a remote provider, an unacceptable trade-off in privacy-sensitive professions and offline or air-gapped settings; the leading on-device alternatives are either platform-locked or aimed at expert scripting rather than plug-and-play dictation. We present YazSes, an open-source (Apache-2.0) hold-to-talk voice dictation daemon that runs entirely on-device, with a single codebase targeting Linux, macOS, and Windows through a protocol-based platform abstraction. YazSes transcribes speech locally with faster-whisper (CPU, int8) and injects the result into the focused application; a fast regex command grammar, backed by an optional small-language-model router, maps utterances to editor and terminal actions. Nothing leaves the machine: recording is push-to-talk rather than always-listening, there is no telemetry, and an opt-in personalization loop keeps its corpus encrypted on-device and proposes configuration changes instead of shipping data out. We describe the system architecture -- a staged pipeline behind a protocol-based platform abstraction with a JSON-RPC control plane -- and its privacy and threat model. We evaluate the shipping Python implementation on a single commodity Linux laptop; the macOS and Windows backends are implemented and unit-tested but not end-to-end evaluated here. On 200 LibriSpeech test-clean utterances spanning 40 speakers, word error rate ranges from 4.82% (tiny.en) to 2.59% (small.en) at a real-time factor of 0.520 for small.en, decoding faster than real time on CPU with no GPU. The command grammar reaches 100% action accuracy with a 0.0% false-positive rate on plain dictation at 0.021 ms per call, and the non-decode pipeline adds 0.289 ms of overhead. The system and the reproducible benchmark harness behind every number in this paper are public.
Innovation
Methods, ideas, or system contributions that make the work stand out.
on-device speech recognition
privacy-preserving voice dictation
cross-platform abstraction
faster-whisper
command grammar
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
