RepliBench: Evaluating the autonomous replication capabilities of language model agents

2025-04-21

AI Summary

This work addresses security risks arising from autonomous replication of large language model (LLM) agents. We propose RepliBench, the first systematic evaluation framework for assessing such capabilities. Methodologically, we decompose autonomous replication into four quantifiable dimensions: resource acquisition, weight theft, cross-platform replication, and long-term persistence. We design 86 novel security evaluation tasks across 20 categories, integrating multimodal techniques—including red-teaming, cloud API interaction, sandboxed program generation, and KYC-bypass simulation—to construct a high-fidelity evaluation environment. Experiments reveal that state-of-the-art models (e.g., Claude 3.7 Sonnet) achieve >50% pass@10 on 15 out of 20 task families; even on the 9 most challenging variants, performance exceeds 50%. These results confirm that component-level replication capabilities are already emergent, and end-to-end autonomous replication is rapidly approaching a critical threshold. RepliBench establishes both a benchmark tool and a theoretical paradigm for evaluating AI agent autonomy and associated security risks.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Large Multimodal Models (LMMs)
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Uncontrollable autonomous replication of language model agents poses a critical safety risk. To better understand this risk, we introduce RepliBench, a suite of evaluations designed to measure autonomous replication capabilities. RepliBench is derived from a decomposition of these capabilities covering four core domains: obtaining resources, exfiltrating model weights, replicating onto compute, and persisting on this compute for long periods. We create 20 novel task families consisting of 86 individual tasks. We benchmark 5 frontier models, and find they do not currently pose a credible threat of self-replication, but succeed on many components and are improving rapidly. Models can deploy instances from cloud compute providers, write self-propagating programs, and exfiltrate model weights under simple security setups, but struggle to pass KYC checks or set up robust and persistent agent deployments. Overall the best model we evaluated (Claude 3.7 Sonnet) has a>50% pass@10 score on 15/20 task families, and a>50% pass@10 score for 9/20 families on the hardest variants. These findings suggest autonomous replication capability could soon emerge with improvements in these remaining areas or with human assistance.

Problem

• Evaluating autonomous replication risks of language model agents
• Measuring capabilities in resource acquisition and model exfiltration
• Assessing persistent deployment threats from frontier AI models

Innovation

• RepliBench evaluates autonomous replication risks
• Tests cover resource acquisition and model exfiltration
• Benchmarks 5 models with 86 task variations