Large Language Models for In-File Vulnerability Localization Can Be"Lost in the End"

2025-02-09

AI Summary

This work systematically evaluates the effectiveness of conversational large language models (LLMs), including GPT-3.5 and GPT-4, in whole-file-level vulnerability localization—specifically for XSS, SQL injection, and path traversal vulnerabilities. Method: Through empirical analysis and two-tailed t-tests, we identify and name the “lost-in-the-end” effect: LLMs exhibit statistically significant performance degradation (p < 0.05) when localizing vulnerabilities near the end of long source files. We further characterize the joint impact of input length and vulnerability position on recall and propose a model-agnostic optimization strategy—optimal-size chunking with sliding windows—based on input-length sensitivity. Contribution/Results: Our strategy improves average recall by 37.2% across models on benchmark datasets. The findings yield both actionable, reproducible guidelines and theoretical insights for LLM-driven security analysis.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Evaluation and Analysis
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Recent advancements in artificial intelligence have enabled processing of larger inputs, leading everyday software developers to increasingly rely on chat-based large language models (LLMs) like GPT-3.5 and GPT-4 to detect vulnerabilities across entire files, not just within functions. This new development practice requires researchers to urgently investigate whether commonly used LLMs can effectively analyze large file-sized inputs, in order to provide timely insights for software developers and engineers about the pros and cons of this emerging technological trend. Hence, the goal of this paper is to evaluate the effectiveness of several state-of-the-art chat-based LLMs, including the GPT models, in detecting in-file vulnerabilities. We conducted a costly investigation into how the performance of LLMs varies based on vulnerability type, input size, and vulnerability location within the file. To give enough statistical power to our study, we could only focus on the three most common (as well as dangerous) vulnerabilities: XSS, SQL injection, and path traversal. Our findings indicate that the effectiveness of LLMs in detecting these vulnerabilities is strongly influenced by both the location of the vulnerability and the overall size of the input. Specifically, regardless of the vulnerability type, LLMs tend to significantly (p<.05) underperform when detecting vulnerabilities located toward the end of larger files, a pattern we call the 'lost-in-the-end' effect. Finally, to further support software developers and practitioners, we also explored the optimal input size for these LLMs and presented a simple strategy for identifying it, which can be applied to other models and vulnerability types. Eventually, we show how adjusting the input size can lead to significant improvements in LLM-based vulnerability detection, with an average recall increase of over 37% across all models.

Problem

• Evaluate LLMs' effectiveness in detecting in-file vulnerabilities
• Investigate 'lost-in-the-end' effect in LLM performance
• Determine optimal input size for LLM vulnerability detection

Innovation

• Evaluates GPT models for vulnerability detection
• Identifies 'lost-in-the-end' effect in LLMs
• Optimizes input size for improved detection