ParEVO: Synthesizing Code for Irregular Data: High-Performance Parallelism through Agentic Evolution

2026-03-02

AI Summary

This work addresses the challenges of generating efficient parallel code for irregular data structures—such as sparse graphs and unbalanced trees—with large language models, which often suffer from race conditions, deadlocks, and suboptimal performance. To overcome these issues, the authors propose a novel approach combining high-quality instruction tuning with an evolutionary coding agent. They construct the semantically aligned Parlay-Instruct corpus, fine-tune models including DeepSeek, Qwen, and Gemini, and integrate compiler feedback, runtime race detection, and Work-Span parallel primitives to automatically repair and optimize the “last-mile” parallel code. Evaluated on the ParEval benchmark, the method achieves an average speedup of 106× (up to 1,103×), delivers 13.6× acceleration on complex graph algorithms, and in some kernels even outperforms expert hand-optimized implementations by up to 4.1×.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Evolutionary Computation

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The transition from sequential to parallel computing is essential for modern high-performance applications but is hindered by the steep learning curve of concurrent programming. This challenge is magnified for irregular data structures (such as sparse graphs, unbalanced trees, and non-uniform meshes) where static scheduling fails and data dependencies are unpredictable. Current Large Language Models (LLMs) often fail catastrophically on these tasks, generating code plagued by subtle race conditions, deadlocks, and sub-optimal scaling. We bridge this gap with ParEVO, a framework designed to synthesize high-performance parallel algorithms for irregular data. Our contributions include: (1) The Parlay-Instruct Corpus, a curated dataset of 13,820 tasks synthesized via a "Critic-Refine" pipeline that explicitly filters for empirically performant algorithms that effectively utilize Work-Span parallel primitives; (2) specialized DeepSeek, Qwen, and Gemini models fine-tuned to align probabilistic generation with the rigorous semantics of the ParlayLib library; and (3) an Evolutionary Coding Agent (ECA) that improves the "last mile" of correctness by iteratively repairing code using feedback from compilers, dynamic race detectors, and performance profilers. On the ParEval benchmark, ParEVO achieves an average 106x speedup (with a maximum of 1103x) across the suite, and a robust 13.6x speedup specifically on complex irregular graph problems, outperforming state-of-the-art commercial models. Furthermore, our evolutionary approach matches state-of-the-art expert human baselines, achieving up to a 4.1x speedup on specific highly-irregular kernels. Source code and datasets are available at https://github.com/WildAlg/ParEVO.

Problem

• irregular data structures
• parallel code synthesis
• concurrent programming
• race conditions
• performance scalability

Innovation

• parallel code synthesis
• irregular data structures
• evolutionary coding agent
• LLM fine-tuning
• ParlayLib