Assessing Spear-Phishing Website Generation in Large Language Model Coding Agents

2026-02-13

AI Summary

This study systematically evaluates the abuse risk of large language model (LLM) coding agents in generating spear-phishing websites. By assessing the capability and willingness of 40 LLM-based agents to autonomously produce phishing sites, the authors construct a dataset comprising 200 website code samples and corresponding interaction logs, enabling a detailed analysis of their threat potential in social engineering attacks. The work presents the first empirical evidence linking specific LLM characteristics to their capacity for generating malicious code, thereby offering critical insights into the security implications of LLM deployment. These findings provide a foundational benchmark, empirical basis, and resource for future research on mitigating the misuse of LLMs in cyber-offensive contexts.

Technology Category

Machine Learning: Large Multimodal Models (LMMs)Natural Language Processing: (Large) Language ModelsMultiagent Systems: Adversarial Agents

Application Category

User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendationEconomics, Online Markets and Human Computation: Cost models of using LLMs in production systemsGraph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Large Language Models are expanding beyond being a tool humans use and into independent agents that can observe an environment, reason about solutions to problems, make changes that impact those environments, and understand how their actions impacted their environment. One of the most common applications of these LLM Agents is in computer programming, where agents can successfully work alongside humans to generate code while controlling programming environments or networking systems. However, with the increasing ability and complexity of these agents comes dangers about the potential for their misuse. A concerning application of LLM agents is in the domain cybersecurity, where they have the potential to greatly expand the threat imposed by attacks such as social engineering. This is due to the fact that LLM Agents can work autonomously and perform many tasks that would normally require time and effort from skilled human programmers. While this threat is concerning, little attention has been given to assessments of the capabilities of LLM coding agents in generating code for social engineering attacks. In this work we compare different LLMs in their ability and willingness to produce potentially dangerous code bases that could be misused by cyberattackers. The result is a dataset of 200 website code bases and logs from 40 different LLM coding agents. Analysis of models shows which metrics of LLMs are more and less correlated with performance in generating spear-phishing sites. Our analysis and the dataset we present will be of interest to researchers and practitioners concerned in defending against the potential misuse of LLMs in spear-phishing.

Problem

• Spear-Phishing
• Large Language Models
• LLM Agents
• Cybersecurity
• Social Engineering

Innovation

• LLM agents
• spear-phishing
• code generation
• cybersecurity
• social engineering