BioTIER: A Refusal Benchmark for Targeted Biological Risk Mitigation

2026-07-15

AI Summary

This work addresses critical safety challenges faced by large language models in the biological domain, where models either risk disclosing hazardous information or overly restrict legitimate scientific inquiries due to a lack of precise risk identification and differentiated control mechanisms. To resolve this, the authors propose BioTIER—the first risk-tiered biosafety evaluation benchmark—which categorizes biological content into three distinct classes: catastrophe-averting, dual-use research of concern, and general biology. Leveraging an expert-curated dataset of 542 metadata-annotated prompts, BioTIER enables accurate discrimination between high-risk information and beneficial scientific knowledge. The benchmark facilitates targeted refusal strategies that effectively block a minimal set of catastrophic content while preserving open access to the vast majority of research-relevant knowledge, thereby significantly enhancing both model safety and scientific utility.

Abstract

As large language models become increasingly capable, concerns about their potential to assist with biological misuse continue to grow. Prioritization of safety differs across the model ecosystem, with some models freely providing high-risk information that could be misused, and others refusing benign scientific content, potentially hindering legitimate research. Both failures stem from a lack of targeted mitigation to distinguish the most dangerous information from broader scientific content. To address this, we introduce BioTIER (Biological Targeted Information for Exclusion and Refusal), a benchmark designed to enable more targeted biological risk mitigation. BioTIER organizes biological content into three risk sets: Catastrophe Avoidance (CA), Biomedical DURC (BD) and Related Biology (RB). These sets represent a spectrum from extremely narrow high-risk topics to a broad range of benign and beneficial biological knowledge. The benchmark consists of 542 expert-curated prompts with rich associated metadata to support differentiated access policies. We release BioTIER to aid in isolating and gating the tiny fraction of information that could engender catastrophic risk from misuse, while ensuring access to the vast wealth of knowledge that is essential for advancing biological science.

Problem

• biological risk mitigation
• refusal benchmark
• dual-use research
• large language models
• targeted safety

Innovation

• BioTIER
• targeted risk mitigation
• refusal benchmark
• dual-use research
• biological safety