ClinLens: Towards Long-Horizon Coding Agents for Longitudinal Multimodal Clinical Data Science

2026-07-28

AI Summary

This work addresses the limitations of existing clinical data science benchmarks in supporting auditable, complex analyses of longitudinal, multimodal electronic health records. To bridge this gap, the authors propose CLINLENS—the first end-to-end executable benchmark tailored for long-term, multimodal clinical data—encompassing five data modalities from MIMIC: structured records, clinical notes, electrocardiograms, chest X-rays, and echocardiograms. The benchmark defines 200 tasks and employs a 4×5 taxonomy that systematically evaluates performance across patient–time scopes and analytical capabilities. It introduces private reference workflows and rigorous semantic validation to distinguish between executability and clinical correctness. Experiments reveal that state-of-the-art models achieve only a 56.3% scope-macro STRICTPASS rate on 126 fixed tasks, while a biomedical system adapted from GPT-4o-mini attains merely 2.9%, underscoring a critical deficiency in clinical reasoning accuracy among current AI agents.

Abstract

Clinical data-science agents must transform heterogeneous longitudinal records into auditable analyses, yet existing benchmarks largely isolate medical question answering, structured-table reasoning, or generic scientific repositories. We introduce CLINLENS, a benchmark of 200 executable tasks over five linked MIMIC resources spanning structured electronic health records, notes, electrocardiograms, chest radiographs, and echocardiograms. A 4 x 5 taxonomy crosses four patient-time scopes with five analysis capabilities. Program-first reverse synthesis pairs each bounded semi-raw package with an evaluator-private reference workflow and checks required artifacts, cohort and temporal semantics, and the final answer. On a fixed 126-task suite, the strongest of 24 standardized model-scaffold configurations achieves 56.3% scope-macro STRICTPASS despite 100% EXECSUCCESS. For reference, a separately configured coding agent solves 83 of 126 tasks, while five biomedical systems adapted to GPT-4o-mini reach at most 2.9% scope-macro STRICTPASS. These results expose a substantial gap between runnable submissions and correct clinical analyses.

Problem

• longitudinal clinical data
• multimodal data
• clinical reasoning
• executable tasks
• benchmarking

Innovation

• longitudinal multimodal clinical data
• executable benchmark
• program-first reverse synthesis
• clinical reasoning agent
• STRICTPASS evaluation