From Human Guidance to Autonomy: Agent Skill System for End-to-End LLM Deployment on Spatial NPUs

2026-05-27

AI Summary

This work addresses the challenge of fully automated end-to-end deployment of large language models (LLMs) on resource-constrained spatial NPUs, a process still hindered by heavy manual intervention and fragmented workflows. The authors propose a two-stage approach: first, a human-in-the-loop reference deployment of Llama-3.2-1B is constructed while recording the optimization trajectory; second, this trajectory is distilled into a structured agent skill system comprising eight stages. This system enables, for the first time, fully automated, generalizable deployment across multiple unseen decoder-only LLMs without human intervention, while rigorously preserving numerical correctness. Evaluated on the AMD XDNA 2 NPU platform, the method successfully deploys eight new models within 0.5–4 hours, all passing numerical validation, with three matching or exceeding the performance of manual reference deployments—demonstrating both efficacy and strong generalization capability.

Abstract

Spatial neural processing units (NPUs) provide an energy-efficient platform for edge LLM inference, but efficiently deploying an LLM end-to-end on such hardware remains labor-intensive. Although AI coding agents have begun to lower this cost, existing studies have largely focused on single-kernel optimization rather than end-to-end LLM deployment on resource-constrained spatial NPUs. We present a two-stage methodology, instantiated on the AMD XDNA 2 NPU, that progresses from human-guided development to agent autonomy. In the first stage, we develop a reference deployment of Llama-3.2-1B through human-guided agent assistance. The resulting implementation achieves a speedup of 2.2x on prefill and 4.0x on decode over the hand-optimized baseline, with the optimization trajectory and its lessons recorded as structured documentation throughout. In the second stage, we distill the documentation into an agent skill system consisting of eight phases, orchestrating the optimization and debugging skill sets, with numerical correctness strictly enforced at each phase. Using our agent skill system, we autonomously deploy eight additional decoder-only LLMs (Llama-3.2-3B, SmolLM2-1.7B, Qwen2.5-{0.5B, 1.5B, 3B}, Qwen3-{0.6B, 1.7B, 4B}) end-to-end on the AMD XDNA 2 NPU using the open-source compiler stack. To our knowledge, these models have not previously been deployed on AMD NPUs via any open-source software stack. Each deployment completes in 0.5-4 hours of agent wall time with almost no human guidance, and passes the numerical-correctness gates, demonstrating functional generalization to previously unencountered LLMs. Three of the eight match or exceed the sustained performance of our Llama-3.2-1B reference deployment, suggesting that the resulting implementations can be competitive without additional model-specific human engineering.

Problem

• Spatial NPU
• LLM deployment
• end-to-end optimization
• autonomous deployment
• edge inference

Innovation

• agent skill system
• end-to-end LLM deployment
• spatial NPU
• autonomous optimization
• numerical correctness