🤖 AI Summary
This study addresses the lack of quantitative comparison between certified compilation and fully formal verification in terms of development cost, performance impact, and practical feasibility. Leveraging the first verified compiler developed by a coding agent under human supervision, the work systematically evaluates both approaches in implementing common optimizations—such as unreachable code elimination, dead assignment elimination, and constant propagation/folding—focusing on development overhead, algorithmic choices, proof complexity, and runtime efficiency. The results show that formal verification incurs an order-of-magnitude higher development cost, leading the agent to favor less efficient algorithms and narrower optimization scopes; certificate checking becomes a performance bottleneck. Nevertheless, with modern coding agents, both methods remain practically feasible for the selected optimizations, offering the first quantitative insights into the key trade-offs imposed by verification, including increased development burden, performance compromises, and heightened supervision requirements.
📝 Abstract
Formal program verification is a longstanding goal in the field. We present the first quantitative comparison of the two primary compiler verification approaches, credible compilation/translation validation and full verification. Working with the first verified compiler developed by a coding agent (operating under human supervision), we present quantitative results from a coding agent implementing several optimizations using these two approaches. The results indicate that 1) verification requires roughly an order of magnitude more development effort than credible compilation, 2) to enhance provability, the coding agent chooses less efficient algorithms and data structures for verified optimizations, and 3) in an attempt to minimize proof effort the coding agent repeatedly implemented optimization scope reductions for verified optimizations, and 4) certificate checking time dominates optimization and certificate generation time for the considered optimizations. Because of the increased proof overhead, verified optimizations required substantially more supervision and coding sessions than credible compilation optimizations. Given the capabilities of a modern coding agent working in this context, implementation efforts for both credible compilation and verified versions remained feasible for the considered optimizations (unreachable code elimination, dead assignment elimination, and constant propagation/folding).