Do LLMs Strategically Reveal, Conceal, and Infer Information? A Theoretical and Empirical Analysis in The Chameleon Game

2025-01-31

AI Summary

This work investigates fundamental deficiencies of large language models (LLMs) in strategic information control tasks—specifically, the “chameleon game,” a linguistic coordination game requiring identity concealment, selective disclosure, and role inference about others. Method: We construct a multi-agent game framework using GPT-4, GPT-4o, Gemini 1.5, and Claude 3.5 Sonnet, integrating game-theoretic modeling with controlled linguistic evaluation. Contribution/Results: We provide the first systematic empirical evidence that all tested LLMs exhibit pervasive over-disclosure in identity-ambiguous settings: as non-chameleon agents, they consistently underperform baseline strategic policies in win rate. Theoretical analysis derives an information disclosure strategy spectrum and tight theoretical bounds on winning probability, confirming systematic deviation from optimal information control. Our work establishes the first verifiable, theory-grounded empirical benchmark for assessing LLM trustworthiness in privacy-sensitive interactive scenarios.

Technology Category

• Game Theory and Economic Paradigms: Imperfect Information
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large language model-based (LLM-based) agents have become common in settings that include non-cooperative parties. In such settings, agents' decision-making needs to conceal information from their adversaries, reveal information to their cooperators, and infer information to identify the other agents' characteristics. To investigate whether LLMs have these information control and decision-making capabilities, we make LLM agents play the language-based hidden-identity game, The Chameleon. In the game, a group of non-chameleon agents who do not know each other aim to identify the chameleon agent without revealing a secret. The game requires the aforementioned information control capabilities both as a chameleon and a non-chameleon. The empirical results show that while non-chameleon LLM agents identify the chameleon, they fail to conceal the secret from the chameleon, and their winning probability is far from the levels of even trivial strategies. To formally explain this behavior, we give a theoretical analysis for a spectrum of strategies, from concealing to revealing, and provide bounds on the non-chameleons' winning probability. Based on the empirical results and theoretical analysis of different strategies, we deduce that LLM-based non-chameleon agents reveal excessive information to agents of unknown identities. Our results point to a weakness of contemporary LLMs, including GPT-4, GPT-4o, Gemini 1.5, and Claude 3.5 Sonnet, in strategic interactions.

Problem

• Large Language Models
• Strategic Information Control
• Chameleon Game

Innovation

• Chameleon Game
• Strategic Interaction
• Language Model Limitations