Feasibility of Time-Domain DNN-Based Speech Enhancement on Embedded FPGA for Hearing Aid

2026-06-02

AI Summary

This work addresses the stringent latency and power constraints of hearing aids for speech enhancement, which existing deep neural networks struggle to meet on embedded platforms. The authors deploy a lightweight SuDoRM-RF++ model on the AMD-Xilinx Kria KV260 FPGA, integrating 16-bit fixed-point quantization, on-chip parameter caching, and hardware acceleration to achieve, for the first time, clinically compliant DNN-based speech enhancement on an FPGA. Experimental results reveal that data movement—not computational throughput—is the primary latency bottleneck. Through precision compression, the model’s memory footprint is halved without degrading speech quality. The proposed accelerator achieves a first-sample latency of 9.7 ms (below the 10 ms clinical threshold) for noise suppression and 16.0 ms for speech separation, thereby clarifying the resource requirements and optimization pathways for embedded deployment.

Abstract

Hearing aids impose strict latency and power constraints that current DNN-based speech enhancement systems struggle to meet on embedded hardware. We characterize this gap by deploying both speech separation and denoising using the lightweight SuDoRM-RF++ architecture on the AMD-Xilinx Kria KV260, evaluated at FP32 and 16-bit fixed-point precision for each task. Across these configurations, first-sample latency tracks with on-chip parameter caching rather than arithmetic throughput, identifying data movement as the primary bottleneck. Precision reduction halves the model memory footprint without compromising objective speech quality. The fixed-point denoising accelerator achieves a first-sample latency of 9.7~ms, meeting the 10~ms clinical threshold, while speech separation reaches 16.0~ms. These measurements establish concrete resource requirements for embedded DNN-based speech enhancement and quantify the remaining gap to hearing aid deployment.

Problem

• speech enhancement
• hearing aid
• embedded FPGA
• latency constraint
• power constraint

Innovation

• embedded FPGA
• DNN-based speech enhancement
• fixed-point quantization
• low-latency inference
• hearing aid