Can LLMs Recover Program Semantics? A Systematic Evaluation with Symbolic Execution

📅 2025-11-24
📈 Citations: 0
✨ Influential: 0
📄 PDF
🤖 AI Summary
Large language models (LLMs) struggle to recover the original semantics of programs obfuscated or optimized by compilers—e.g., via control-flow flattening, opaque predicates, or arithmetic/branch encoding—hindering program understanding, testing, and vulnerability analysis. To address this, we propose a symbolic-execution-augmented LLM semantic recovery method: leveraging KLEE to generate SMT constraints, path coverage statistics, and concrete test cases, we construct high-fidelity training data to fine-tune an LLM for joint modeling of syntactic structure and semantic constraints. This work is the first to systematically inject precise semantic information from symbolic execution into LLM training. Experiments on our custom obfuscation benchmark show that after fine-tuning, GPT-4.1-mini achieves a 37.2% improvement in compilation success rate and 89.6% behavioral equivalence—substantially outperforming state-of-the-art text-only or static-analysis baselines.

Technology Category

Machine Learning: Large Multimodal Models (LMMs)Natural Language Processing: Code Generation / Program Synthesis from Natural LanguageSearch and Optimization: Learning to Search

Application Category

Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMsSearch and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved informationEconomics, Online Markets and Human Computation: Cost models of using LLMs in production systems
📝 Abstract
Obfuscation poses a persistent challenge for software engineering tasks such as program comprehension, maintenance, testing, and vulnerability detection. While compiler optimizations and third-party code often introduce transformations that obscure program intent, existing analysis tools and large language models (LLMs) struggle to recover the original semantics. In this work, we investigate whether LLMs, when fine-tuned with symbolic execution artifacts, can effectively deobfuscate programs and restore analyzability. We construct a benchmark by applying four widely studied transformations-control-flow flattening, opaque predicates, arithmetic encoding, and branch encoding-across diverse C programs from TUM Obfuscation Benchmarks, the LLVM test suite, and algorithmic repositories. We then compare three state-of-the-art LLMs under two training configurations: baseline fine-tuning on obfuscated/original code pairs, and enhanced fine-tuning with additional KLEE artifacts such as SMT constraints, path statistics, and test cases. Our evaluation examines syntactic correctness (compilation success), semantic fidelity (behavioral equivalence under symbolic execution), and code quality (readability and structure). Results show that GPT-4.1-mini achieves the strongest deobfuscation overall, and that incorporating KLEE artifacts consistently improves semantic preservation and compilation success across models. These findings highlight deobfuscation as a broader software engineering concern, demonstrating that combining LLMs with symbolic execution can strengthen automated testing, static analysis, and program comprehension in the presence of obfuscation.
Problem

Research questions and friction points this paper is trying to address.

Evaluating LLMs' ability to recover program semantics from obfuscated code
Investigating symbolic execution-enhanced fine-tuning for program deobfuscation
Assessing semantic preservation and compilation success across obfuscation transformations
Innovation

Methods, ideas, or system contributions that make the work stand out.

Fine-tuning LLMs with symbolic execution artifacts
Using KLEE artifacts like SMT constraints for deobfuscation
Combining LLMs with symbolic execution for program analysis
R
Rong Feng
The Pennsylvania State University, USA
S
Suman Saha
The Pennsylvania State University, USA