The Influence of Human-inspired Agentic Sophistication in LLM-driven Strategic Reasoners

2025-05-14

AI Summary

This paper investigates whether LLM-based agents can replicate human strategic reasoning, specifically examining how human-inspired cognitive structures influence LLMs’ game-theoretic reasoning capabilities. Method: Using the number-guessing game as a testbed, we systematically evaluate three structured agent architectures—incorporating memory, planning, and reflection modules—via game-theoretic modeling and a controlled experimental platform across 25 configurations and over 2,000 human-agent comparison samples. Contribution/Results: Human-inspired architectural priors significantly improve alignment with human strategy distributions but yield limited out-of-distribution generalization, constrained by inherent LLM capabilities. Agent complexity exhibits a nonlinear relationship with human behavioral similarity: merely stacking modules degrades performance. We propose a new paradigm—“cognitive architecture alignment over mechanical augmentation”—advocating lightweight, interpretable, and capability-aware design that leverages LLMs’ intrinsic strengths rather than imposing rigid, biologically unmotivated structures.

Technology Category

• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Mechanism Design
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

The rapid rise of large language models (LLMs) has shifted artificial intelligence (AI) research toward agentic systems, motivating the use of weaker and more flexible notions of agency. However, this shift raises key questions about the extent to which LLM-based agents replicate human strategic reasoning, particularly in game-theoretic settings. In this context, we examine the role of agentic sophistication in shaping artificial reasoners' performance by evaluating three agent designs: a simple game-theoretic model, an unstructured LLM-as-agent model, and an LLM integrated into a traditional agentic framework. Using guessing games as a testbed, we benchmarked these agents against human participants across general reasoning patterns and individual role-based objectives. Furthermore, we introduced obfuscated game scenarios to assess agents' ability to generalise beyond training distributions. Our analysis, covering over 2000 reasoning samples across 25 agent configurations, shows that human-inspired cognitive structures can enhance LLM agents' alignment with human strategic behaviour. Still, the relationship between agentic design complexity and human-likeness is non-linear, highlighting a critical dependence on underlying LLM capabilities and suggesting limits to simple architectural augmentation.

Problem

• Assessing LLM-based agents' replication of human strategic reasoning
• Evaluating agentic sophistication in artificial reasoners' performance
• Exploring limits of architectural augmentation in human-like behavior

Innovation

• Integrates LLMs into traditional agentic frameworks
• Uses human-inspired cognitive structures for alignment
• Tests generalization with obfuscated game scenarios