Data Aware Differentiable Neural Architecture Search for Tiny Keyword Spotting Applications

2025-07-21

AI Summary

To address the high design complexity and the trade-off between resource constraints and performance in TinyML systems, this paper proposes a data-aware differentiable neural architecture search (DARTS) method. It innovatively incorporates data configuration parameters—such as feature extraction methods and data augmentation strategies—into the differentiable search space, enabling, for the first time, end-to-end joint optimization of model architecture and data preprocessing. Leveraging gradient-based co-search, the method automatically discovers lightweight, efficient models for edge tasks like keyword spotting. Compared to baseline approaches, the resulting models achieve significant reductions in model size and computational cost while maintaining or even improving recognition accuracy. This work establishes a novel automated design paradigm for TinyML systems that simultaneously enhances efficiency, accuracy, and usability.

Technology Category

• Machine Learning: Hardware-aware ML
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

The success of Machine Learning is increasingly tempered by its significant resource footprint, driving interest in efficient paradigms like TinyML. However, the inherent complexity of designing TinyML systems hampers their broad adoption. To reduce this complexity, we introduce "Data Aware Differentiable Neural Architecture Search". Unlike conventional Differentiable Neural Architecture Search, our approach expands the search space to include data configuration parameters alongside architectural choices. This enables Data Aware Differentiable Neural Architecture Search to co-optimize model architecture and input data characteristics, effectively balancing resource usage and system performance for TinyML applications. Initial results on keyword spotting demonstrate that this novel approach to TinyML system design can generate lean but highly accurate systems.

Problem

• Optimize neural architecture and data for TinyML
• Balance resource usage and performance efficiently
• Design lean, accurate keyword spotting systems

Innovation

• Expands search space to data configuration parameters
• Co-optimizes model architecture and input data
• Balances resource usage and system performance