An Empirical Security Evaluation of LLM-Generated Cryptographic Rust Code

2026-04-28

AI Summary

This study addresses the critical security risks in cryptographic code generated by large language models, which often evade detection by existing general-purpose verification tools. The authors present the first set of cryptography-specific vulnerability detection rules tailored for AES-256-GCM and ChaCha20-Poly1305, integrating CodeQL with a custom-built analyzer. Their approach identifies vulnerabilities in 57% of compilable samples with zero false positives, though only 23.3% of generated code successfully compiles. Through empirical evaluation across multiple models and prompting strategies, they find that chain-of-thought prompting significantly underperforms zero-shot prompting (p = 0.002) and uncover prevalent flaws such as nonce reuse and API hallucination. These findings underscore the necessity of domain-specific validation mechanisms in cryptographic contexts.

Abstract

Developers and organizations are using Large Language Models (LLMs) to generate security-critical code more frequently than ever, including cryptographic solutions for their products. This study presents an empirical evaluation of cryptographic security in 240 Rust code samples for two crypto algorithms (AES-256-GCM and ChaCha20-Poly1305) generated by three LLMs (Gemini 2.5 Pro, GPT-4o, and DeepSeek Coder) using four different prompt strategies. For each successfully compiled code sample, CodeQL static analysis and our rule-based crypto-specific analyzer were used to detect vulnerabilities, which are also associated with Common Weakness Enumeration (CWE). The evaluation results revealed that only 23.3% of the generated code samples were successfully compiled. Among the compiled code, CodeQL produced only two false positives, while our rule-based crypto-specific analyzer identified vulnerabilities in 57% of the compiled samples with zero false positives. This demonstrates that general-purpose analysis tools are insufficient for code validation for the experimented crypto algorithms. The compilation success of the two algorithms varied significantly (AES-256-GCM 34.2% versus ChaCha20-Poly1305 12.5%), showing a gap in code generation capabilities. While model choice had no significant effect on compilation success, prompt strategy significantly influenced outcomes (P = 0.002), with chain-of-thought prompting performing 5 times worse than zero-shot. All three models exhibit systematic failures, including nonce reuse and API hallucinations.

Problem

• Large Language Models
• Cryptographic Code
• Security Vulnerabilities
• Rust
• Code Generation

Innovation

• LLM-generated cryptographic code
• empirical security evaluation
• crypto-specific static analysis
• prompt strategy impact
• Rust cryptography