🤖 AI Summary
To address the challenge of efficiently deploying large language models (LLMs) on resource-constrained mobile and edge devices, this paper systematically surveys and unifies key model compression techniques—including knowledge distillation, quantization (INT4/FP8), pruning (structured and unstructured), mixture-of-experts (MoE), and early-exit—within a single analytical framework. We introduce the first comprehensive taxonomy covering over 120 methods, characterizing their accuracy–latency–power trade-offs and delineating practical applicability boundaries. Methodologically, we propose two novel research directions: hardware-software co-designed compression and dynamic sparse inference, which substantially enhance the composability and synergy of existing techniques. Our contributions include an open, reusable technology selection guide and an empirical benchmark suite grounded in real-world edge deployment scenarios. This work provides both theoretical foundations and actionable engineering insights for efficient LLM deployment at the edge.
📝 Abstract
Large Language Models (LLMs) have revolutionized many areas of artificial intelligence (AI), but their substantial resource requirements limit their deployment on mobile and edge devices. This survey paper provides a comprehensive overview of techniques for compressing LLMs to enable efficient inference in resource-constrained environments. We examine three primary approaches: Knowledge Distillation, Model Quantization, and Model Pruning. For each technique, we discuss the underlying principles, present different variants, and provide examples of successful applications. We also briefly discuss complementary techniques such as mixture-of-experts and early-exit strategies. Finally, we highlight promising future directions, aiming to provide a valuable resource for both researchers and practitioners seeking to optimize LLMs for edge deployment.