CAPS: A Cascaded Reconstruction Model to Power Saving in Hearables Using Sub-Nyquist Sampling with Bandwidth Extension

2026-07-20

AI Summary

This work addresses the challenge of balancing power consumption and speech quality in hearing-assistive wearable devices when reducing ADC sampling rates and bit precision, as well as the lack of effective narrowband-to-wideband speech reconstruction methods. The authors propose CAPS, a cascaded reconstruction model that, for the first time in ear-worn devices, integrates sub-Nyquist sampling, low-bit quantization, and a lightweight streaming bandwidth extension architecture to enable efficient speech reconstruction. The proposed approach achieves a 3.3× reduction in power consumption, with an inference latency of only 1.36 ms and a memory footprint of 11.04 MB, while maintaining high speech intelligibility in real-world scenarios—effectively bridging the gap between low-power operation and high-quality speech reconstruction.

Abstract

Hearables are wearable computers worn on the ear. Bone conduction microphones are used with air conduction microphones in hearables for multimodal speech enhancement in noisy conditions. Despite this potential, current models largely fail to explore how jointly reducing sampling bit resolution and sampling frequency in analog-to-digital converters (ADCs) of hearables impacts both power usage and audio quality. Furthermore, current frameworks cannot do sub-Nyquist sampling in hearables because they lack a method to reconstruct wideband signals from narrowband components. We therefore propose CAPS, which (i) intentionally employs sub-Nyquist sampling and low bit resolution in ADCs, achieving a 3.3x reduction in power consumption in hearables, and (ii) supports streaming operation on mobile platforms with an inference time of 1.36 ms and a memory footprint of 11.04 MB. CAPS ensures robust speech intelligibility in real-world settings, bridging the gap between efficiency and power savings.

Problem

• hearables
• sub-Nyquist sampling
• power consumption
• bandwidth extension
• speech intelligibility

Innovation

• sub-Nyquist sampling
• bandwidth extension
• low-bit ADC
• speech reconstruction
• power-efficient hearables