Fast Data Aware Neural Architecture Search via Supernet Accelerated Evaluation

2025-02-18

AI Summary

In ultra-resource-constrained TinyML scenarios, conventional neural architecture search (NAS) methods optimize only model topology, neglecting the critical impact of input data configurations (e.g., resolution, sampling rate) on system-level efficiency and accuracy. Method: This paper proposes a joint data–architecture optimization framework, introducing the first data-aware differentiable NAS paradigm. It incorporates input configuration parameters into the searchable space and designs a hardware-aware, multi-objective supernet evaluation mechanism to enable efficient co-search of both data preprocessing and neural architecture. Contribution/Results: Evaluated on the Wake Vision dataset, our method significantly outperforms architecture-only NAS baselines across diverse time and hardware constraints (e.g., latency, energy budget), achieving superior accuracy–efficiency trade-offs. It establishes a novel system-level co-optimization paradigm for TinyML, enabling end-to-end optimization spanning data representation, model structure, and hardware deployment constraints.

Technology Category

• Machine Learning: Hardware-aware ML
• Search and Optimization: Learning to Search
• Computer Vision: Learning & Optimization for CV

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Tiny machine learning (TinyML) promises to revolutionize fields such as healthcare, environmental monitoring, and industrial maintenance by running machine learning models on low-power embedded systems. However, the complex optimizations required for successful TinyML deployment continue to impede its widespread adoption. A promising route to simplifying TinyML is through automatic machine learning (AutoML), which can distill elaborate optimization workflows into accessible key decisions. Notably, Hardware Aware Neural Architecture Searches - where a computer searches for an optimal TinyML model based on predictive performance and hardware metrics - have gained significant traction, producing some of today's most widely used TinyML models. Nevertheless, limiting optimization solely to neural network architectures can prove insufficient. Because TinyML systems must operate under extremely tight resource constraints, the choice of input data configuration, such as resolution or sampling rate, also profoundly impacts overall system efficiency. Achieving truly optimal TinyML systems thus requires jointly tuning both input data and model architecture. Despite its importance, this"Data Aware Neural Architecture Search"remains underexplored. To address this gap, we propose a new state-of-the-art Data Aware Neural Architecture Search technique and demonstrate its effectiveness on the novel TinyML ``Wake Vision'' dataset. Our experiments show that across varying time and hardware constraints, Data Aware Neural Architecture Search consistently discovers superior TinyML systems compared to purely architecture-focused methods, underscoring the critical role of data-aware optimization in advancing TinyML.

Problem

• Optimize TinyML via Data Aware Neural Architecture Search
• Jointly tune input data and model architecture
• Enhance TinyML system efficiency under constraints

Innovation

• Data Aware Neural Architecture Search
• Joint tuning of input data
• Supernet Accelerated Evaluation technique