HLSTester: Efficient Testing of Behavioral Discrepancies with LLMs for High-Level Synthesis

2025-04-20

AI Summary

Detecting behavioral discrepancies between C/C++ source programs and their synthesized hardware circuits in high-level synthesis (HLS), arising from hardware-specific characteristics, remains challenging. Method: We propose the first LLM-driven automated testing framework for HLS. It integrates backward slicing to identify critical variables, LLM-guided testbench generation, and a synergistic input-generation mechanism combining dynamic mutation with progressive reasoning chains. A redundancy-aware hardware test-skipping strategy further enhances efficiency. The framework unifies program analysis, LLM prompt engineering, runtime spectrum monitoring, and redundancy filtering. Contribution/Results: Evaluated on multiple HLS benchmarks, our framework improves simulation pass rate by 32% over conventional methods and by 27% over direct LLM-based approaches. It significantly reduces manual intervention and shortens test cycles while maintaining rigorous behavioral validation.

Technology Category

• Machine Learning: Hardware-aware ML
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

In high-level synthesis (HLS), C/C++ programs with synthesis directives are used to generate circuits for FPGA implementations. However, hardware-specific and platform-dependent characteristics in these implementations can introduce behavioral discrepancies between the original C/C++ programs and the circuits after high-level synthesis. Existing methods for testing behavioral discrepancies in HLS are still immature, and the testing workflow requires significant human efforts. To address this challenge, we propose HLSTester, a large language model (LLM) aided testing framework that efficiently detects behavioral discrepancies in HLS. To mitigate hallucinations in LLMs and enhance prompt quality, the testbenches for original C/C++ programs are leveraged to guide LLMs in generating HLS-compatible testbenches, effectively eliminating certain traditional C/C++ constructs that are incompatible with HLS tools. Key variables are pinpointed through a backward slicing technique in both C/C++ and HLS programs to monitor their runtime spectra, enabling an in-depth analysis of the discrepancy symptoms. To reduce test time, a testing input generation mechanism is introduced to integrate dynamic mutation with insights from an LLM-based progressive reasoning chain. In addition, repetitive hardware testing is skipped by a redundancy-aware filtering technique for the generated test inputs. Experimental results demonstrate that the proposed LLM-aided testing framework significantly accelerates the testing workflow while achieving higher testbench simulation pass rates compared with the traditional method and the direct use of LLMs on the same HLS programs.

Problem

• Detects behavioral discrepancies in high-level synthesis (HLS) implementations
• Reduces human effort in testing C/C++ to FPGA circuit conversions
• Mitigates LLM hallucinations for accurate HLS testbench generation

Innovation

• LLM-guided testbench generation for HLS compatibility
• Backward slicing to pinpoint key variables
• Dynamic mutation with LLM-based reasoning chain