GPUPhysBench: Benchmarking Coding Agents for Correct and Efficient GPU Physics Simulation

2026-09-28

AI Summary

This study addresses the challenge that coding agents struggle to simultaneously achieve numerical accuracy and execution efficiency when generating GPU-based physical simulation code. To investigate this, we introduce physics simulation into agent evaluation for the first time by constructing a benchmark comprising 50 fluid and solid simulation tasks. Under a fixed time budget, we assess the ability of agents to write, compile, and optimize NVIDIA GPU code involving techniques such as collision detection and iterative solvers. Our experiments reveal that although the strongest model completes all tasks, only 22% of its solutions attain 90% of the reference execution speed, with significant performance gaps observed in collision handling and constraint resolution. These findings expose the core bottlenecks of current coding agents in high-performance computational mechanics.

Abstract

Writing fast GPU code for physical simulation is difficult: implementations must preserve numerical accuracy while handling irregular data access, synchronization, and iterative solvers. We introduce GPUPhysBench, a benchmark of 50 tasks testing whether coding agents can meet these demands. Tasks cover fluids, deformable solids, and granular materials, from individual simulation operators to complete simulators. Agents write, compile, test, and optimize GPU code with access to a NVIDIA GPU under fixed time budgets. We report pass rates and runtime performance relative to expert-optimized reference implementations. In a single-attempt evaluation of six frontier model-harness pairs, the two strongest pass all 50 tasks, but even the fastest reaches at least 0.9 the reference speed on only 22% of them, and no submission is more than 5% faster than the reference. The largest gaps arise in collision detection, constraint solving, and iterative solvers. GPUPhysBench brings physical simulation workloads to coding-agent evaluation, testing both the ability to implement numerical methods correctly and the ability to make them run efficiently.

Problem

• GPU Physics Simulation
• Coding Agents
• Benchmarking
• Numerical Accuracy
• Runtime Performance

Innovation

• GPUPhysBench
• Coding Agents
• GPU Physics Simulation
• Benchmarking
• Numerical Methods