Large Language Models Show Signs of Alignment with Human Neurocognition During Abstract Reasoning

2025-08-12

AI Summary

This study investigates whether the abstract reasoning mechanisms of large language models (LLMs) align with human neural cognition. Method: Using an abstract pattern completion task, we simultaneously recorded fixation-related potentials (FRPs) from human participants and analyzed the geometric structure of intermediate-layer representations across eight open-source LLMs (7B–72B parameters), employing cross-modal representational similarity analysis. Contribution/Results: Only 70B-scale models (Qwen-2.5-72B, DeepSeek-R1-70B) achieved human-level behavioral performance. All models exhibited task-relevant abstract category clustering in intermediate layers, with clustering strength positively correlated with accuracy. Critically, the optimal representational layer in these models showed moderate geometric similarity (r ≈ 0.45) to prefrontal FRPs in representational space. This work provides the first empirical evidence of structural commonality between human and LLM representational spaces in abstract reasoning—offering foundational support for grounding AI cognitive interpretability in neurobiological principles.

Technology Category

Machine Learning: Large Multimodal Models (LMMs)Natural Language Processing: (Large) Language ModelsCognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactionsUser Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendationEconomics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

This study investigates whether large language models (LLMs) mirror human neurocognition during abstract reasoning. We compared the performance and neural representations of human participants with those of eight open-source LLMs on an abstract-pattern-completion task. We leveraged pattern type differences in task performance and in fixation-related potentials (FRPs) as recorded by electroencephalography (EEG) during the task. Our findings indicate that only the largest tested LLMs (~70 billion parameters) achieve human-comparable accuracy, with Qwen-2.5-72B and DeepSeek-R1-70B also showing similarities with the human pattern-specific difficulty profile. Critically, every LLM tested forms representations that distinctly cluster the abstract pattern categories within their intermediate layers, although the strength of this clustering scales with their performance on the task. Moderate positive correlations were observed between the representational geometries of task-optimal LLM layers and human frontal FRPs. These results consistently diverged from comparisons with other EEG measures (response-locked ERPs and resting EEG), suggesting a potential shared representational space for abstract patterns. This indicates that LLMs might mirror human brain mechanisms in abstract reasoning, offering preliminary evidence of shared principles between biological and artificial intelligence.

Problem

• Do LLMs mirror human neurocognition in abstract reasoning
• Compare human and LLM performance on abstract-pattern tasks
• Assess shared representational space between human and LLM

Innovation

• Compares LLMs and human neurocognition via EEG
• Largest LLMs match human accuracy and patterns
• LLM layers cluster abstract categories like humans