Technology Category
Machine Learning: Hardware-aware MLSearch and Optimization: Learning to SearchCognitive Modeling & Cognitive Systems: Agent Architectures
Optimizing edge AI models on HPC systems with the edge in the loop
🤖 AI Summary
Deploying compact AI models on edge devices—such as those used in additive manufacturing—faces a fundamental trade-off among high accuracy, low latency, and minimal memory footprint. To address this, we propose an “edge-in-the-loop” hardware-aware neural architecture search (NAS) paradigm. Our approach integrates real-time latency measurements from physical edge hardware deployed in Belgium into a NAS workflow orchestrated on a high-performance computing (HPC) platform in Germany, establishing a cross-regional collaborative optimization loop. It further incorporates distributed heterogeneous training and fine-tuning on the RAISE-LPBF dataset. Compared to manually designed baselines, the discovered architectures achieve an 8.8× speedup in inference latency and a 1.35× improvement in quality-related accuracy metrics. This work significantly advances the feasibility and practicality of end-to-end co-optimization for edge AI models.
Technology Category
Application Category
📝 Abstract
Artificial intelligence and machine learning models deployed on edge devices, e.g., for quality control in Additive Manufacturing (AM), are frequently small in size. Such models usually have to deliver highly accurate results within a short time frame. Methods that are commonly employed in literature start out with larger trained models and try to reduce their memory and latency footprint by structural pruning, knowledge distillation, or quantization. It is, however, also possible to leverage hardware-aware Neural Architecture Search (NAS), an approach that seeks to systematically explore the architecture space to find optimized configurations. In this study, a hardware-aware NAS workflow is introduced that couples an edge device located in Belgium with a powerful High-Performance Computing system in Germany, to train possible architecture candidates as fast as possible while performing real-time latency measurements on the target hardware. The approach is verified on a use case in the AM domain, based on the open RAISE-LPBF dataset, achieving ~8.8 times faster inference speed while simultaneously enhancing model quality by a factor of ~1.35, compared to a human-designed baseline.
Problem
Research questions and friction points this paper is trying to address.
Optimizing edge AI models for speed and accuracy
Leveraging hardware-aware NAS for efficient architecture search
Coupling edge devices with HPC for real-time training
Innovation
Methods, ideas, or system contributions that make the work stand out.
Hardware-aware Neural Architecture Search (NAS)
Edge-HPC coupling for real-time latency
Optimized edge AI models via NAS
🔎 Similar Papers
No similar papers found.
