ABC-Bench: An Agentic Bio-Capabilities Benchmark for Biosecurity

2026-06-09

AI Summary

This study systematically evaluates the capabilities of large language model (LLM) agents in biosafety-relevant tasks, balancing their scientific potential against misuse risks. We introduce the first benchmark suite encompassing both benign and dual-use biological tasks, requiring models to integrate biological knowledge with programming skills to perform high-risk operations such as liquid-handling robot control, DNA fragment design, and circumvention of synthetic screening protocols. Crucially, we incorporate wet-lab validation to assess real-world feasibility. Results demonstrate that all evaluated models surpass the median performance of human experts, with scripts generated by o4-mini-high successfully executing DNA assembly on a physical robotic platform—providing the first empirical evidence of end-to-end LLM agent feasibility in complex biological experimentation.

Abstract

Large language models (LLMs) are rapidly acquiring capabilities relevant to biological research, from literature synthesis to interpretation of experimental data. Increasingly, LLM agents can also perform in silico biology tasks that previously required experienced human biologists. These emerging AI capabilities offer new opportunities for scientific discovery and biomedical advances, but they also shift the landscape of biosecurity risks. To address this, we introduce the Agentic Bio-Capabilities Benchmark (ABC-Bench), a suite of tasks to measure agentic biosecurity-relevant capabilities. ABC-Bench evaluates LLM agents on both benign and dual-use biology tasks: writing code to operate liquid handling robots, designing DNA fragments for in vitro assembly, and evading DNA synthesis screening. These tasks require a combination of biology and software expertise. All tested LLM agents outperformed the median expert human baseliner on all three tasks. Agents performed highly on tasks drawing on published knowledge and well-documented protocols, and more weakly on a task requiring novel bioinformatics reasoning. In three wet-lab validation experiments, we found that OpenAI's o4-mini-high produced scripts that, when run on an OpenTrons liquid handling robot, successfully assembled DNA with expected sequences.

Problem

• biosecurity
• large language models
• agentic capabilities
• dual-use biology
• AI safety

Innovation

• agentic benchmarking
• biosecurity
• large language models
• in silico biology
• wet-lab validation