🤖 AI Summary
This work addresses the accuracy degradation of nonlinear activation functions in hardware implementations caused by quantization and truncation. To tackle this issue, the authors propose the Full-space Quantization-driven Architecture (FQA), which, for the first time, comprehensively models the coefficient space affected by quantization. By decoupling the bit-widths across different computational stages and integrating a target-guided binary-window (TBW) acceleration algorithm, FQA efficiently searches for optimal piecewise polynomial approximation coefficients. The architecture supports flexible and configurable hardware mapping. When applied to the Sigmoid function, FQA reduces area and power consumption by over 50% compared to the state-of-the-art, significantly decreases the number of segments, and achieves the best-reported maximum absolute error (MAE) performance.
📝 Abstract
In this paper, we propose a full-space quantization-driven architecture (FQA) for the hardware-efficient piecewise polynomial approximations (PPAs) of nonlinear activation functions. FQA comprehensively considers both fractional-bit truncation error and quantization error that cause the deviation of the optimal approximation coefficients. Crucially, FQA can precisely determine and search the complete range of optimal coefficients. Based on the proposed FQA, we develop two distinct hardware implementation schemes to cater to different resource-performance trade-offs. Furthermore, we decouple all the fractional word lengths (FWLs) involved in the calculation process to enable the exploration of superior hardware architectures. To mitigate the increased software computation time caused by the expanded quantization space, we design an acceleration method named TBW (target-guided bisection window) to expedite the piecewise calculation and searching process. Experimental results demonstrate that, compared to existing architectures, FQA can significantly reduce the number of required segments while achieving the optimal Maximum Absolute Error (MAE). For the hardware design of the Sigmoid function, our approach achieves over 50% reduction in area and power consumption compared to the state-of-the-art PPA architecture. Finally, we present a complete design workflow for deploying PPA on configurable hardware, maximizing the utilization of existing hardware resources and minimizing MAE.