Keyword Spotting with Hyper-Matched Filters for Small Footprint Devices

2025-08-06

AI Summary

To address open-vocabulary keyword spotting on resource-constrained devices, this paper proposes a lightweight and efficient framework. It employs Tiny Whisper or Tiny Conformer as the speech encoder and introduces a character-level hypernetwork-based keyword encoder. A novel hyper-matching filter dynamically generates keyword-specific convolutional weights conditioned on the character sequence, while a Perceiver-based cross-attention detection network enables end-to-end matching. The core contribution is the hyper-matching filter: it synthesizes compact, keyword-adaptive convolutional kernels without increasing inference latency, thereby significantly improving generalization across domains and for non-native (L2) speech. The smallest variant contains only 4.2 million parameters yet achieves state-of-the-art accuracy across multiple out-of-domain scenarios—outperforming baseline models with several times more parameters.

Technology Category

• Natural Language Processing: Speech
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Open-vocabulary keyword spotting (KWS) refers to the task of detecting words or terms within speech recordings, regardless of whether they were included in the training data. This paper introduces an open-vocabulary keyword spotting model with state-of-the-art detection accuracy for small-footprint devices. The model is composed of a speech encoder, a target keyword encoder, and a detection network. The speech encoder is either a tiny Whisper or a tiny Conformer. The target keyword encoder is implemented as a hyper-network that takes the desired keyword as a character string and generates a unique set of weights for a convolutional layer, which can be considered as a keyword-specific matched filter. The detection network uses the matched-filter weights to perform a keyword-specific convolution, which guides the cross-attention mechanism of a Perceiver module in determining whether the target term appears in the recording. The results indicate that our system achieves state-of-the-art detection performance and generalizes effectively to out-of-domain conditions, including second-language (L2) speech. Notably, our smallest model, with just 4.2 million parameters, matches or outperforms models that are several times larger, demonstrating both efficiency and robustness.

Problem

• Detecting untrained keywords in speech recordings
• Achieving high accuracy on small-footprint devices
• Generalizing to out-of-domain conditions like L2 speech

Innovation

• Uses hyper-network for keyword-specific matched filters
• Employs tiny Whisper or Conformer as speech encoder
• Integrates Perceiver module with cross-attention mechanism