R-GroundBench: A Diagnostic Benchmark for R-Group Groundingin Markush Molecular Editing

2026-09-30

AI Summary

This study addresses the limitation of existing benchmarks in evaluating AI reasoning for patent molecular editing, particularly regarding R-group grounding in Markush structures. We construct the first diagnostic benchmark targeting incomplete chemical representations derived from real patents. By employing multimodal visual question answering and open-ended generation tasks with a controlled difficulty stratification strategy, this work systematically evaluates models’ cross-modal comprehension and execution capabilities for variable R-groups. Our approach effectively distinguishes superficial recognition from deep reasoning, revealing critical limitations in domain-pretrained models. Experimental results demonstrate that model accuracy degrades significantly in complex scenarios, with extremely low generation success rates, confirming that current AI systems lack reliable Markush editing capabilities.

Abstract

Recent advances in AI for scientific discovery enable molecular understandingand design, yet reasoning over incomplete chemical representations remainsunclear.Markush structures, which encode molecular families through variable R-groupplaceholders (\textit{R\textsubscript{1}}, \textit{R\textsubscript{2}}, \textit{X}, etc.), are ubiquitous in pharmaceutical patents and requiregrounding across molecular, textual, and chemical information.However, existing molecule-language benchmarks focus on fully specifiedmolecules, leaving R-group grounding largely unevaluated.We introduce R-GroundBench:, a diagnostic benchmark built from real patent Markushstructures, featuring a Multiple-Choice (VQA) track with controlled difficultyand modality splits, and an open-ended Generation track.Our results reveal a substantial gap between recognition andmolecular grounding.While models achieve over 90\% accuracy on Easy VQA, performance drops to56--66\% on Hard VQA when shortcuts are controlled.Chemical-domain VLMs also remain unreliable, achieving only 25.7--46.2\% on HardVQA despite domain-specific pretraining.Moreover, Generation Exact Match remains below 20\% for most models and below8\% when visual input is required.These findings reveal that current AI systems lack reliable grounding andexecution for Markush editing, highlighting challenges for AI-drivenscientific discovery.

Problem

• Markush structures
• R-group grounding
• molecular editing
• benchmark evaluation
• AI for science

Innovation

• Markush structures
• R-group grounding
• diagnostic benchmark
• visual question answering
• molecular editing