MiCoPro: End-to-End Mixed Precision HW/SW Co-design with HW-aware Proxy Model

2026-08-07

AI Summary

This work addresses the limitations of existing mixed-precision quantization methods, which often struggle to balance model accuracy, hardware acceleration efficiency, and end-to-end deployability. To overcome these challenges, we propose MiCoPro, a framework that integrates a hardware-aware surrogate model to efficiently search for high-accuracy mixed-precision configurations under strict latency constraints. MiCoPro enables automatic deployment from PyTorch models directly to bare-metal C code, combining quantization-aware training with target-hardware latency modeling. The framework is designed to support diverse hardware platforms. Experimental results demonstrate that on both BitFusion accelerators and SIMD-enhanced RISC-V processors, MiCoPro reduces latency by up to 40% while incurring less than 3% accuracy loss, significantly outperforming state-of-the-art approaches.

Abstract

Quantized Neural Networks~(QNN) with low-bitwidth data have proven promising in efficient storage and computation on edge devices. To mitigate accuracy degradation while maximizing speedup, layer-wise mixed-precision quantization~(MPQ) becomes a popular solution. However, existing algorithms for exploring MPQ schemes are limited in flexibility and efficiency. Comprehending the complex impacts of different MPQ schemes on post-training quantization and quantization-aware training results is a challenge for conventional methods. Furthermore, an end-to-end framework for the optimization and deployment of MPQ models is missing in existing work. To address these challenges, we propose the MiCo framework, a holistic MPQ exploration and deployment framework for edge AI applications. The framework adopts a novel optimization algorithm to search for accuracy-optimal quantization configurations under strict latency constraints. We further extended the framework to MiCoPro, which introduces a robust Hardware-Aware Proxy (HAP) model to enhance prediction accuracy and hardware versatility. By leveraging target-specific latency modeling, MiCoPro enables rapid exploration and direct deployment from PyTorch models to bare-metal C code. We demonstrate the versatility of our framework on both the BitFusion accelerator and SIMD-extended RISC-V processors, achieving up to 40\% of latency reduction with less than 3\% of accuracy drop.

Problem

• Mixed-Precision Quantization
• Quantized Neural Networks
• Hardware-Aware Optimization
• End-to-End Deployment
• Edge AI

Innovation

• Mixed-Precision Quantization
• Hardware-Aware Proxy
• End-to-End Deployment
• Quantization-Aware Training
• Edge AI