A Blind Spot in Alignment: Quantifying Biosecurity Risks in Large Language Models

2026-08-02

AI Summary

Current safety evaluations of large language models are confined to natural language and fail to identify genuine biological hazards in generated amino acid sequences, creating significant security blind spots. This work proposes SPIKE-Bench, the first functional biosafety evaluation benchmark tailored for protein sequences, comprising 631 toxin-design prompts and a three-stage SPIKE funnel that assesses model outputs along dimensions of compliance, biological plausibility, and predicted toxicity. Introducing the Functional Harmfulness Rate (FHR) to quantify real-world risk, the study demonstrates that model refusal rates do not correlate with actual biological danger and presents BioSafe-Guard, a domain-specific classifier. Audits of 32 large models reveal that most freely generate high-risk sequences (FHR up to 50.7%), whereas BioSafe-Guard substantially mitigates such risks while preserving benign functionalities.

Abstract

Large Language Models (LLMs) are accelerating biological research, yet this same capability poses a critical biosecurity threat: models that assist in protein engineering can equally be prompted to generate predicted toxin-like sequences, potentially lowering the barrier to biological misuse. Current safety evaluations, however, operate in natural language and cannot determine whether a model-generated amino acid sequence is biological gibberish or a computational risk signal. To address this evaluation blind spot, we introduce SPIKE-Bench, coupling 631 curated toxin-design prompts across seven functional categories with the SPIKE funnel, a three-stage protocol that filters output through compliance, biological plausibility, and predicted toxicity, producing stage-level diagnostics and an aggregate function-aware metric: the Functional Harmfulness Rate (FHR). An audit of 32 LLMs reveals that most models freely comply with toxin-design requests; FHR is driven primarily by biological generation capability rather than safety alignment, reaching 50.7%; and Refusal Rate fails to predict functional risk. As a first step toward mitigation, we provide BioSafe-Guard, a domain-specialized classifier that substantially reduces predicted functional risk while preserving benign utility. We release SPIKE-Bench and BioSafe-Guard at https://github.com/PKU-Alignment/SPIKE-Bench to support more rigorous biosecurity evaluation of LLMs.

Problem

• biosecurity
• large language models
• toxin design
• biological risk
• safety evaluation

Innovation

• biosecurity
• large language models
• toxin design
• Functional Harmfulness Rate
• domain-specialized safety