Operational Robustness of LLMs on Code Generation

2026-02-21

AI Summary

Existing approaches struggle to evaluate the robustness of large language models in code generation tasks against minor perturbations in natural language inputs, primarily due to the discrete nature of the input space and the absence of systematic metrics. This work proposes a “scenario-domain analysis” framework—the first discrete natural language robustness evaluation methodology tailored for code generation—that systematically assesses model robustness by identifying minimal semantic changes that induce erroneous outputs. Integrating formal verification with multidimensional scenario categorization (e.g., task complexity and programming topic), the framework enables rigorous operational robustness evaluation. Experiments across Gemini-Pro, Codex, Llama2, and Falcon 7B not only yield theoretically grounded robustness rankings but also reveal significant performance degradation in complex tasks and advanced programming domains such as multithreading and data structures.

Technology Category

Natural Language Processing: Safety and RobustnessMachine Learning: Large Multimodal Models (LMMs)Computer Vision: Adversarial Attacks & Robustness

Application Category

Search and Retrieval-Augmented AI: Web evaluation methodologies and metricsEconomics, Online Markets and Human Computation: Cost models of using LLMs in production systemsWeb Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

It is now common practice in software development for large language models (LLMs) to be used to generate program code. It is desirable to evaluate the robustness of LLMs for this usage. This paper is concerned in particular with how sensitive LLMs are to variations in descriptions of the coding tasks. However, existing techniques for evaluating this robustness are unsuitable for code generation because the input data space of natural language descriptions is discrete. To address this problem, we propose a robustness evaluation method called scenario domain analysis, which aims to find the expected minimal change in the natural language descriptions of coding tasks that would cause the LLMs to produce incorrect outputs. We have formally proved the theoretical properties of the method and also conducted extensive experiments to evaluate the robustness of four state-of-the-art art LLMs: Gemini-pro, Codex, Llamma2 and Falcon 7B, and have found that we are able to rank these with confidence from best to worst. Moreover, we have also studied how robustness varies in different scenarios, including the variations with the topic of the coding task and with the complexity of its sample solution, and found that robustness is lower for more complex tasks and also lower for more advanced topics, such as multi-threading and data structures.

Problem

• robustness
• code generation
• large language models
• natural language descriptions
• operational robustness

Innovation

• scenario domain analysis
• code generation
• LLM robustness
• natural language perturbation
• discrete input space