Humans Are More Diverse: Frontier LLMs Show Extreme Policies in Idealised AI Development Races

2026-08-02

AI Summary

This study addresses the tension between safety and speed in AI development competitions by modeling them as multi-agent repeated games, where strategic safety behaviors are essential. The authors construct a multi-player repeated game environment and systematically evaluate the strategic rationality of large language models through a multi-level auditing framework—encompassing rule recall, state tracking, payoff computation, and trajectory analysis under diverse prompting and decoding strategies. Integrating evolutionary game-theoretic benchmarks with human experimental data, the work reveals that state-of-the-art models exhibit highly divergent and extreme strategies under identical rules. While some models can accurately recite the rules, they often fail to correctly compute payoffs or maintain consistent state representations. Model behavior proves highly sensitive to implementation details and response formats, with arithmetic reasoning capabilities exerting a pronounced influence—an effect obscured by aggregate statistics that mask critical individual-level strategic heterogeneity.

Abstract

An AI development race creates a multi-agent safety dilemma. Each company can develop slowly and safely, or move faster while taking a risk that may remove its final reward. We use this repeated game to study strategic safety behaviour among large language model (LLM) agents in races with two to five players. However, a valid action does not show that an agent understands the game. We therefore place an audit gate before behavioural interpretation. We first verify the game engine, then test rule recall, state tracking, payoff calculation, and stability under different but equivalent task descriptions. We then compare LLM action sequences with an evolutionary game-theory benchmark and published human data, and explore differences across models, risk conditions, personas, and two- to five-player races. The audit shows that strong rule recall can coexist with weak state tracking and expected-payoff calculation. Providing verified arithmetic and changing the response representation can also change later actions, even when the game rules stay fixed. Across seven tested model endpoints, aggregate rates hide large differences in action sequences, responses to opponents, and responses to race position. Patterns across the tested three- to five-player races are also model-specific rather than a single effect of adding competitors. These results show why multi-agent AI-race simulations need validity checks and trajectory-level analysis before their outputs are described as strategic, human-like, or safety-aware. Our findings are exploratory and apply only to the tested models, prompts, and decoding settings.

Problem

• AI development race
• multi-agent safety dilemma
• large language models
• strategic behavior
• validity verification

Innovation

• audit framework
• multi-agent safety dilemma
• trajectory-level analysis
• LLM strategic behavior
• evolutionary game theory