Cloud-Boosted Low-Compute Multi-Channel Speech Enhancement

2026-08-07

AI Summary

This work addresses the challenge of achieving high-performance real-time speech enhancement on resource-constrained wearable devices under stringent latency requirements. To overcome this limitation, the authors propose a novel cloud-edge collaborative speech enhancement framework that innovatively integrates intermediate feature representations and covariance matrices from both edge and server models. By strategically delaying server outputs, performing layer-wise feature enhancement, and applying collaborative multi-channel Wiener filtering, the framework enables efficient cooperative inference. Remarkably, this approach achieves substantial improvements in speech enhancement quality over edge-only baseline models while incurring negligible additional computational overhead on the edge device.

Abstract

Low-latency, low-compute speech enhancement is essential for wearable devices with real-time communication requirements, but strict computational constraints significantly limit on-device performance. Knowledge Boosting has been proposed as an effective approach to improve edge model performance by leveraging a more capable server-side model, but performance gains for speech enhancement have been limited. We propose a collaborative framework incorporating three techniques: (1) delayed server output as additional input, (2) layerwise feature boosting that transfers intermediate server representations to guide edge inference, and (3) collaborative multichannel Wiener filtering, which fuses weighted covariance matrices estimated from both server and edge models for improved beamforming. Experimental results demonstrate that the proposed collaborative framework significantly outperforms the edge-only baseline with minimal additional computational overhead.

Problem

• speech enhancement
• low-compute
• wearable devices
• real-time communication
• computational constraints

Innovation

• Cloud-Boosted
• Low-Compute Speech Enhancement
• Layerwise Feature Boosting
• Collaborative Multichannel Wiener Filtering
• Edge-Server Collaboration