From Code Review to Code Critique: Intent, Drift, and Spotlight for AI-Generated Diffs at Scale

2026-07-31

AI Summary

The surge in AI-generated code has rendered traditional manual code review unsustainable, while existing tools often prioritize low-value suggestions and overlook critical aspects such as correctness, security, and performance. This work proposes ARCTIC, a novel framework that integrates developer intent prediction (leveraging conversation logs and metadata), generation drift detection (via back-translation), and difference-aware code focusing (guided by quality estimation) to enable efficient review of AI-generated code. ARCTIC uniquely combines these techniques with a six-theme taxonomy, substantially enhancing the identification of high-risk changes. Offline evaluations demonstrate an intent prediction F1 score of 0.86, a drift detection quadratically weighted kappa (QWK) of 0.907, and a 2.4× improvement in quality assessment efficacy when focusing on only one-fifth of tokens. Online experiments show that drift scoring effectively reduces code misalignment, intent predictions are endorsed by 90.2% of developers, and no self-reviewed defect reports emerged post-deployment.

Abstract

AI coding agents are generating code at volumes that exceed the capacity of traditional peer review. At the same time, existing AI code review tools over-index on low-value suggestions such as style and best practices while under-indexing on the concerns human reviewers prioritize most: correctness, security, and performance. We present ARCTIC, an AI-powered Code Critique system that reframes code review around three capabilities: intent prediction, which infers why a change was made from conversation logs and metadata; drift detection, which measures divergence between the developer's intent and the agent's output via backtranslation; and code spotlight, which ranks the regions of a diff most warranting human scrutiny. We ground these capabilities in a six-theme taxonomy derived from 18,000 code reviews. Offline evaluation shows that intent prediction achieves 0.86 F1, drift detection reaches near-perfect ordinal agreement with human annotators (QWK = 0.907), and spotlight outperforms the baseline AI reviewer by 2.4x on quality estimation at 5x fewer tokens. In the experimental rollout, the drift scores reduces code misalignment by an additional 5.76 points (p = 0.026), intent prediction receives 90.2% approval, and zero defects have been attributed to self-reviewed diffs since launch.

Problem

• code review
• AI-generated code
• correctness
• security
• performance

Innovation

• intent prediction
• drift detection
• code spotlight
• AI code critique
• backtranslation