numeric representation optimization

To address the high computational cost and low hardware efficiency of floating-point operations in deep learning training, this paper proposes a hardware-aware low-precision logarithmic fixed-point training method tailored for accelerators. The approach innovatively incorporates a bit-width–aware mechanism into logarithmic addition approximation, jointly optimizing piecewise linear approximation and simulated annealing to achieve Pareto-optimal trade-offs between accuracy and hardware overhead. Leveraging the logarithmic number system (LNS) and bit-accurate C++ simulation, end-to-end training is realized using 12-bit integer arithmetic. Experimental results on VGG-11 and VGG-16 demonstrate accuracy comparable to 32-bit floating-point training, while reducing multiply-accumulate (MAC) unit area by 32.5% and energy consumption by 53.5%. This work establishes a novel hardware–software co-design paradigm for low-precision deep learning training.

High-level synthesis (HLS) tools—such as Chisel and commercial HLS compilers—often produce circuits with inferior performance compared to hand-designed hardware in high-performance computing (HPC) accelerator design. Method: This paper proposes a hierarchical algorithmic decomposition and automated evaluation framework that abstracts mathematical kernels (e.g., Fourier transforms, matrix multiplication, QR decomposition) into reusable building blocks, uniformly implemented across multiple abstraction levels (RTL, Chisel, C++ HLS), and systematically benchmarked for resource utilization, timing, and operating frequency. Contribution/Results: The framework establishes the first cross-abstraction-level fair benchmarking methodology, enabling fine-grained identification of inefficiencies introduced by HLS compilers. Experimental evaluation demonstrates significantly improved accuracy and interpretability in pinpointing design bottlenecks, providing quantitative guidance for both HLS tool optimization and practical hardware design.

This work addresses the lack of systematic, multi-granularity comparison between low-precision floating-point (FP) and integer (INT) quantization for handling LLM activation outliers on AI hardware (e.g., Blackwell architecture). We conduct algorithm–hardware co-design, proposing three techniques: fine-grained block-wise quantization, Hadamard rotation to suppress outliers, and symmetric clipping to eliminate gradient bias in low-bit training. We首次 demonstrate that MXINT8 achieves near-lossless training at 4 bits—surpassing MXFP4/NVFP4 in both accuracy and energy efficiency. Moreover, NVINT4, when combined with outlier control, outperforms NVFP4. These findings challenge the FP-dominant paradigm in AI hardware design, establishing a unified evaluation framework for low-bit quantization and introducing an integer-first, co-optimized path for efficient LLM inference and training.

Whether 16-bit floating-point (FP16) arithmetic alone can robustly support end-to-end neural network training under hardware resource constraints remains a long-standing, systematically unverified question. Method: This paper introduces the first unified framework for classification-tolerant error analysis, integrating theoretical floating-point error modeling with large-scale empirical validation, and implements fully FP16 forward and backward propagation—without any FP32 “master weights” or loss scaling. Contribution/Results: Rigorous evaluation on CIFAR-10, CIFAR-100, and ImageNet demonstrates that pure FP16 training achieves accuracy parity with FP32 or mixed-precision baselines (accuracy deviation ≤ ±0.1%), while accelerating training by 1.8×. All experiments deploy seamlessly on mainstream GPUs with zero code modification. This work bridges a critical gap between theoretical guarantees and practical feasibility of low-precision training.

Existing hardware multipliers struggle to efficiently support the dynamic precision requirements of mixed-precision quantized neural networks at runtime, leading to a trade-off between resource utilization and model accuracy. To address this challenge, this work proposes a runtime-reconfigurable, multi-precision, multi-channel bit-level systolic array architecture that, for the first time, enables dynamic inter-layer mixed-precision multiplication. By integrating bit-level systolic arrays, runtime reconfiguration mechanisms, and multi-channel parallel processing, the proposed architecture achieves 1.32–3.57× inference acceleration on an Ultra96 FPGA, with reduced critical-path delay and support for operating frequencies up to 250 MHz. This design significantly enhances model accuracy adaptability while maintaining high hardware efficiency.

Deploying deep neural networks on resource-constrained embedded systems presents significant challenges in simultaneously achieving energy efficiency, correctness, and security—particularly in safety-critical domains such as healthcare, where models are vulnerable to fault injection attacks and lack dynamic accuracy guarantees. To address this, this work proposes a hardware-software co-design approach. On the software side, it introduces the first adaptive precision quantization method that jointly ensures real-time operation, dynamic adjustability, and formal correctness, leveraging sensitivity analysis to stabilize decision boundaries. On the hardware side, it designs a customized systolic array supporting left-to-right, most-significant-bit (MSB)-first computation to enhance resilience against bit-flip errors. Experimental results demonstrate the feasibility of the proposed framework, with preliminary findings indicating substantial improvements in both reliability and resource efficiency for edge AI deployments.

This work addresses the prevalent overuse of double-precision floating-point numbers in numerical programs by proposing an automated mixed-precision tuning methodology. The approach supports user-defined, non-standard low-precision floating-point formats with customizable exponent and mantissa bit-widths, and integrates numerical validation with systematic search within a unified framework to automatically generate program variants that meet prescribed accuracy constraints. Leveraging the PROMISE tool and containerized parallel benchmarking, the method demonstrates that numerous variables across a range of numerical applications and the Rodinia benchmark suite can be safely downgraded in precision. This reduction yields significant improvements in performance while simultaneously decreasing memory consumption and energy usage, all without compromising numerical accuracy.

This work addresses the challenge of balancing dynamic range and precision in numerical representation for memory-constrained embedded and edge AI systems. The authors propose WINT, a weighted integer format that enables flexible trade-offs between precision and range through configurable allocation of mantissa and exponent bits at design time. They derive an analytical model for average relative error and introduce, for the first time, a floating-point-like representation supporting user-defined bit widths. Efficient error evaluation is achieved via O(1)-complexity approximations based on harmonic series and Taylor expansions. Experiments demonstrate that with word lengths of 12 bits or more, WINT using just 2 exponent bits reduces average relative error by 12–33% compared to integer baselines while doubling the dynamic range; with 3 exponent bits, it further extends the range by up to 16× and lowers error by 15–50%.

Existing large language models struggle to balance performance and accuracy under uniform low-precision floating-point quantization. This work proposes dMX, the first differentiable mixed-precision quantization framework for the OCP-defined MXFP format. By compressing discrete per-layer bit-width search into a single continuous learnable offset and integrating temperature-annealing scheduling with target-aware regularization, dMX enables a smooth transition from training to hardware deployment. Experiments on Llama, Qwen3, and SmolLM2 demonstrate that dMX consistently outperforms KL-divergence-based heuristic methods across WikiText-2 perplexity and four zero-shot reasoning tasks, achieving Pareto-optimal trade-offs between accuracy and bit-width.

This work addresses the challenge of efficiently implementing very-large-constant multiplication (VLCM) on resource-constrained hardware, overcoming the limitation of existing approaches that are restricted to moderately sized constants. The authors propose an enhanced decompose-optimize-reconstruct pipeline that introduces overlapping patterns to reduce the number of multiple constant multiplication (MCM) subproblems. They formulate a declarative optimization model grounded in constraint programming and SAT solving techniques to achieve globally optimal reconstructions across all stages. This approach yields the first globally optimal solutions for VLCM while precisely delineating their applicability boundaries. Empirical evaluations on signal processing and cryptographic benchmarks spanning tens to thousands of bits demonstrate superior scalability and consistently outperform state-of-the-art baselines.