Fast, Fine-Grained Equivalence Checking for Neural Decompilers

2025-01-08

AI Summary

Current neural decompiler evaluation methods rely on coarse-grained similarity scores and lack instruction-level equivalence verification, hindering precise error localization and interpretability of prediction quality. To address this, we propose CodeAlign—the first instruction-level equivalence alignment framework grounded in formal semantic modeling. It rigorously defines equivalence relations between intra-function code segments and variable names. Leveraging symbolic execution for auxiliary verification, CodeAlign enables fine-grained, interpretable correctness assessment, outputting structured equivalence mappings rather than a single dimensionless score. Experiments demonstrate that CodeAlign significantly improves evaluation accuracy and debuggability, establishing a novel paradigm for trustworthy verification of neural decompilers.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Neural decompilers are machine learning models that reconstruct the source code from an executable program. Critical to the lifecycle of any machine learning model is an evaluation of its effectiveness. However, existing techniques for evaluating neural decompilation models have substantial weaknesses, especially when it comes to showing the correctness of the neural decompiler's predictions. To address this, we introduce codealign, a novel instruction-level code equivalence technique designed for neural decompilers. We provide a formal definition of a relation between equivalent instructions, which we term an equivalence alignment. We show how codealign generates equivalence alignments, then evaluate codealign by comparing it with symbolic execution. Finally, we show how the information codealign provides-which parts of the functions are equivalent and how well the variable names match-is substantially more detailed than existing state-of-the-art evaluation metrics, which report unitless numbers measuring similarity.

Problem

• Neural Decoder Verification
• Prediction Accuracy
• Instruction-Level Code Comparison

Innovation

• codealign
• neural decompiler
• instruction-level comparison