Device-First Feedback: Toward Mobile-Native LLM-Driven Neural Architecture Search

2026-07-29

AI Summary

This work addresses a critical gap in large language model (LLM)-driven neural architecture search for mobile deployment, where existing approaches prioritize GPU accuracy while neglecting essential constraints such as INT8 quantization, delegate selection, and on-device latency, often yielding impractical models. To overcome this, the authors propose the first “device-first” framework that integrates real-device performance feedback into an LLM-based architecture generation loop, establishing a fully automated end-to-end pipeline. This pipeline encompasses QLoRA fine-tuning for architecture proposal, GPU evaluation, INT8 TensorFlow Lite export, automatic delegate selection, on-device benchmarking, and a gating mechanism for dynamic training corpus expansion with support for interruption recovery. Experiments demonstrate that the method achieves a 25.6× improvement in mobile deployment score on CIFAR-10 within the first round (reaching 46.9% quantized accuracy), further revealing a non-monotonic relationship between GPU accuracy and on-device performance.

Abstract

Deploying convolutional neural networks generated by large language models (LLMs) on real mobile hardware requires more than GPU validation accuracy: INT8 TensorFlow Lite export, delegate selection, and on-device latency jointly determine whether a model is usable. We present an automated mobile deployment pipeline that closes the loop from QLoRA fine-tuning of an architecture-generating LLM through GPU evaluation, INT8 export, and physical-device benchmarking to gated augmentation of the training corpus. The pipeline is fully scripted and runs cycle-by-cycle without manual intervention, with resume support after interruptions. We evaluate the same frozen protocol on two benchmarks, CIFAR-10 and CIFAR-100, on a Samsung SM-P613 tablet (seed 42, 20 models per cycle, cycles 0-6). On CIFAR-10, cycle 1 is gate-accepted and improves the mobile deployment score approximately 25.6x over the baseline with a mean quantized accuracy of 46.9%; later cycles raise GPU accuracy but fail the non-decreasing mobile gate. On CIFAR-100, the pre-QLoRA baseline retains the best mobile score; iterative rounds improve GPU accuracy (up to 26.2%) yet cannot surpass cycle 0 on-device, and the training pool stalls at 19 examples after the first accepted round. Together, the two studies show that closed-loop GPU fine-tuning does not guarantee monotonic mobile gains, especially on harder classification tasks, and that multi-dataset, on-device measurement is needed to stress-test deployment objectives. We release per-cycle metrics with 95% confidence intervals, all figures, and complete reproduction commands.

Problem

• mobile deployment
• neural architecture search
• on-device latency
• LLM-driven
• quantization

Innovation

• Device-First Feedback
• LLM-driven NAS
• on-device benchmarking
• INT8 quantization
• closed-loop deployment