efficient backbone design

To address the high computational overhead and memory bottlenecks hindering Vision Transformer (ViT) deployment on edge devices, this paper presents a systematic survey of lightweighting and acceleration techniques tailored for edge scenarios—spanning model compression (e.g., pruning, quantization, knowledge distillation, attention simplification), software optimization (e.g., compiler frameworks such as TVM), and hardware adaptation (e.g., GPU/TPU/FPGA mapping). Its key contributions include: (1) proposing the first unified taxonomy for ViT edge deployment, explicitly characterizing trade-offs among accuracy, latency, power consumption, and hardware platforms; (2) establishing a structured evaluation framework covering 120+ works to identify real-world deployment bottlenecks; and (3) delivering a reproducible, cross-platform technical selection guide to advance co-optimization of accuracy, latency, and power efficiency.

This work addresses the inefficiency of existing vision backbones on low-parallelism hardware such as CPUs, which are typically optimized for highly parallel accelerators. The authors propose design principles tailored for CPU deployment, emphasizing a balance between high multiply-accumulate operations per second (MACpS) and low latency, and introduce CPUBone—the first family of vision backbones explicitly optimized for CPUs. By incorporating grouped convolutions and small kernel sizes, CPUBone reduces computational load while enhancing execution efficiency on CPU hardware. Experiments demonstrate that CPUBone achieves state-of-the-art accuracy–speed trade-offs across diverse CPU platforms and exhibits strong transfer performance on downstream tasks including object detection and semantic segmentation.

This work addresses the lack of systematic methodologies in model optimization, which often relies on heuristic choices and struggles to accommodate diverse deployment constraints. It formalizes model compression and acceleration as a constraint-aware multi-objective engineering decision problem, establishing a unified and actionable framework grounded in five key dimensions: data availability, latency, memory footprint, accuracy tolerance, and retraining budget. By integrating techniques such as quantization, pruning, knowledge distillation, parameter-efficient fine-tuning (PEFT), and inference optimization, the study proposes tailored optimization pipelines for four representative industrial scenarios, delivering a reproducible and quantifiable guide for technology selection.

This work addresses the significant discrepancy between traditional MACs-based efficiency metrics for vision backbones and actual inference latency on edge devices, which hinders hardware-efficient design. By analyzing the divergence between theoretical MACs and real-world execution times of common building blocks, the study identifies key factors governing hardware efficiency. It proposes LowFormer, a novel backbone featuring the lightweight Lowtention module as a replacement for multi-head self-attention. Through hardware-aware co-design of macro- and micro-architectures alongside cross-platform deployment optimizations, LowFormer achieves higher ImageNet accuracy while substantially outperforming state-of-the-art models in speed across diverse hardware platforms—including both edge and desktop GPUs—and demonstrates strong performance on downstream tasks such as classification, detection, and segmentation.

Edge inference systems face poor model adaptability and high reconfiguration overhead due to resource constraints and dynamic environmental changes. To address this, we propose a runtime environment-aware dynamic pruning framework. Our approach features: (1) a novel post-deployment robust pruning–aware training strategy, enabling models to sustain accuracy under varying pruning configurations after deployment; and (2) a node-level adaptive pruning decision algorithm guided by real-time bottleneck monitoring, supporting “prune-on-demand” model slicing and load-aware balancing within distributed inference pipelines. This framework overcomes the limitations of conventional offline static pruning and costly runtime model reconfiguration. Evaluated on a Raspberry Pi 4B cluster, our method achieves a 1.5× improvement in inference throughput and a 3× increase in SLO compliance rate, while preserving model accuracy with no statistically significant degradation.

To address compilation optimization challenges for dynamically shaped models—particularly large language models (LLMs)—on heterogeneous backends, this paper proposes the first unified compilation abstraction enabling cross-level fusion across computational graphs, loop-level tensor programs, and external library calls. Our method introduces: (1) first-class symbolic shape annotations for global dynamic shape tracking, and (2) a multi-level intermediate representation (IR) fusion framework that integrates symbolic shape inference with shape-aware optimization scheduling. Evaluated on multiple GPU architectures, our approach achieves state-of-the-art performance. Moreover, it enables, for the first time, efficient end-to-end deployment of mainstream LLMs on resource-constrained platforms—including smartphones, embedded devices, and web browsers—without model retraining or structural modification. This significantly broadens the practical applicability of dynamically shaped models in real-world edge and client-side scenarios.

This work addresses the challenges of deploying Vision-Language-Action (VLA) models on edge robots, where real-time performance, cost, and energy constraints are critical, yet existing evaluations predominantly rely on desktop GPUs and overlook the potential of heterogeneous edge accelerators. The authors propose a model-hardware co-characterization methodology to construct the first cross-accelerator VLA performance benchmark, revealing a two-stage bottleneck: compute-intensive visual-language backbones and memory-intensive action experts. To mitigate these bottlenecks, they introduce DP-Cache and V-AEFusion optimization strategies that enable asynchronous pipelined parallelism. Experiments demonstrate speedups of 2.9× on GPUs and up to 6× on edge NPUs, with only marginal degradation in task success rates, thereby validating the feasibility of efficient, low-cost VLA deployment on resource-constrained edge platforms.

This study addresses the memory bottleneck in fine-tuning pretrained vision models on low-memory GPUs (e.g., 2 GB). The authors systematically evaluate five parameter-efficient fine-tuning (PEFT) methods and three gradient checkpointing strategies across diverse vision and vision-language models, assessing their accuracy, energy efficiency, and memory consumption. They propose a memory-budget-aware adaptive gradient checkpointing algorithm and extend the NetScore metric to incorporate deployment constraints. Experimental results demonstrate that QLoRA and BitFit reduce energy consumption by 20–30% with only a 1–2% accuracy drop; the adaptive checkpointing strategy lowers peak GPU memory usage by 43–79%; and self-supervised models such as DINOv2 surpass full fine-tuning performance on CIFAR-100 while consuming significantly fewer computational resources.

This work addresses the challenges of deploying large language models under constraints of memory, latency, and hardware cost, where existing post-training compression methods lack a unified and efficient solution for algorithm selection, precision allocation, and hardware adaptation. We propose an open-source, hardware-aware automated compression framework that enables end-to-end model compression with a single command. The framework features automatic model analysis, mixed-precision planning, and staged progressive quantization—from layers to blocks to the entire model. Innovatively, it establishes the first quantized checkpoint as a deployable baseline, ensuring all subsequent optimizations incrementally improve performance on the same model. This approach bridges algorithmic research and production deployment, significantly reducing resource overhead while preserving model accuracy, thereby enhancing the reproducibility and practicality of compression strategies.

Existing large-scale model training systems struggle to flexibly compose diverse parallelization strategies, often relying on manual expert tuning and lacking generality. This work proposes a programmable distributed training system that enables users to declaratively specify composite parallelism strategies—such as data, pipeline, and expert parallelism—through model annotations and scheduling directives. These specifications are compiled via a unified intermediate representation (IR) into device-level execution plans, fully decoupling strategy definition from runtime execution over a global compute-communication DAG. The system is the first to support automatic compilation of user-defined composite strategies, matching the performance of established approaches like ZeRO while significantly improving both performance and memory efficiency in complex scenarios such as DeepSeek-V3’s DualPipe.

Existing edge AI inference systems are constrained by model-level mapping strategies, which hinder efficient utilization of heterogeneous computing resources to accommodate diverse operator characteristics. This work proposes the first unified operator-level scheduling framework that dynamically assigns each operator to the optimal processing unit (CPU/GPU/NPU) based on empirical performance profiling. By constructing a weighted execution graph and solving a shortest-path problem, the framework enables latency- or energy-efficiency-oriented scheduling. It transcends conventional limitations by uniformly supporting sequential execution, intra-model parallelism, and multi-model concurrency, all without relying on model-specific heuristics, thus achieving model-agnostic applicability. Experiments on an Intel Core Ultra SoC demonstrate up to 1.60× speedup with intra-model parallelism, a geometric mean acceleration of 3.42× for concurrent multi-model execution, and an average energy saving of 48.2% under energy-efficient scheduling.