Vul-LMGNNs: Fusing language models and online-distilled graph neural networks for code vulnerability detection

2024-04-23

Information Fusion

AI Summary

Existing code vulnerability detection methods face two key limitations: graph neural networks (GNNs) suffer from restricted receptive fields due to local neighborhood aggregation, hindering long-range structural dependency modeling; while code large language models (code LLMs) excel in semantic understanding, their synergy with GNNs remains underexplored. This paper proposes LLM-GNN, a novel collaborative framework that unifies LLM-driven semantic comprehension with lightweight, online-distilled GNN-based structural modeling in an end-to-end architecture. We introduce a dynamic graph structure online distillation mechanism tailored for vulnerability detection, enabling efficient knowledge transfer over fused abstract syntax tree (AST) and control flow graph (CFG) representations. Evaluated on multiple benchmarks, our method achieves an F1-score of 92.7%, outperforming state-of-the-art approaches by an average of 4.3 percentage points, while accelerating inference by 2.1× and significantly reducing deployment overhead.

Technology Category

Machine Learning: Large Multimodal Models (LMMs)Natural Language Processing: Code Generation / Program Synthesis from Natural LanguageComputer Vision: Large Vision Models

Application Category

Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphsSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMsSearch and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Problem

• Enhancing code vulnerability detection by fusing language models and graph neural networks
• Overcoming limited structural information propagation in traditional GNN approaches
• Improving collaboration between codeLMs and GNNs for semantic-structural analysis

Innovation

• Fuses pre-trained codeLMs with GNNs
• Uses online knowledge distillation mechanism
• Implements implicit-explicit joint training