The Illusion of Secure LLM Code: Closing the Security Gap via Iterative Reprompting

2026-07-26

AI Summary

This work addresses the critical security gaps in identity authentication code automatically generated by large language models, which often lack essential protective mechanisms. The authors propose a dual-mode evaluation framework integrating static analysis and dynamic penetration testing, systematically assessing five leading AI programming assistants against the NIST SP 800-63B standard under four prompting strategies: baseline, security-enhanced, NIST-guided, and iterative re-prompting. The key innovation lies in the iterative re-prompting mechanism, which enables context-aware self-auditing by the model, overcoming the inherent limitations of single-turn prompting in establishing defense-in-depth. Experimental results demonstrate that this approach significantly improves the compliance and security of generated code, particularly in resistance to brute-force attacks, session management, and password handling.

Abstract

Large Language Models (LLMs) are increasingly integrated into software development workflows, yet their ability to autonomously generate secure authentication code remains uncertain. This paper evaluates the security architecture of authentication systems generated by five prominent AI coding assistants through a bi-modal assessment framework combining static code analysis and dynamic penetration testing, mapped to NIST SP 800-63B guidelines. The study examines model behavior across four prompting strategies Basic, Secure, NIST-Based, and Reprompting to reflect varying levels of developer guidance. Empirical results demonstrate that code generated from functional or generically secure prompts consistently omits critical protections, particularly concerning brute-force resistance, session management, and robust password handling. While providing explicit, single-shot NIST context significantly improves compliance, the findings reveal that this remains structurally inadequate. Instead, iterative Reprompting: forcing models into a contextual self-auditing loop is strictly required to achieve a comprehensive, defense-in-depth security architecture. Ultimately, this study proves that current AI coding assistants do not produce secure-by-default applications, dictating that enterprise deployments must transition from single-shot prompt engineering to continuous, standards-driven verification pipelines.

Problem

• Large Language Models
• Secure Code Generation
• Authentication Security
• NIST SP 800-63B
• AI Coding Assistants

Innovation

• Iterative Reprompting
• Secure Code Generation
• LLM Security
• NIST SP 800-63B
• Defense-in-Depth