How Deep is Love in LLMs' Hearts? Exploring Semantic Size in Human-like Cognition
🤖 AI Summary
This study investigates whether large language models (LLMs) possess human-like “semantic size” perception—the cross-modal association between abstract concepts (e.g., “love”) and concrete physical dimensions. Addressing this problem, we employ metaphor association analysis, representational similarity analysis (RSA) of neural activations, comparative evaluation across multimodal large language models (MLLMs), and simulated e-commerce browsing behavior grounded in real-world web shopping patterns. Our contribution is the first systematic definition and quantification of a semantic size cognition dimension in LLMs. Results demonstrate that multimodal pretraining significantly enhances human-aligned performance on this dimension; moreover, semantic size modulates affective decision biases—e.g., large-size framing (e.g., “epic discount”) increases click-through propensity but also susceptibility to misdirection. These findings establish semantic size as a novel cognitive dimension for LLM modeling and underscore the critical role of multimodal experience in grounding abstract concept understanding.
Application Category
📝 Abstract
How human cognitive abilities are formed has long captivated researchers. However, a significant challenge lies in developing meaningful methods to measure these complex processes. With the advent of large language models (LLMs), which now rival human capabilities in various domains, we are presented with a unique testbed to investigate human cognition through a new lens. Among the many facets of cognition, one particularly crucial aspect is the concept of semantic size, the perceived magnitude of both abstract and concrete words or concepts. This study seeks to investigate whether LLMs exhibit similar tendencies in understanding semantic size, thereby providing insights into the underlying mechanisms of human cognition. We begin by exploring metaphorical reasoning, comparing how LLMs and humans associate abstract words with concrete objects of varying sizes. Next, we examine LLMs' internal representations to evaluate their alignment with human cognitive processes. Our findings reveal that multi-modal training is crucial for LLMs to achieve more human-like understanding, suggesting that real-world, multi-modal experiences are similarly vital for human cognitive development. Lastly, we examine whether LLMs are influenced by attention-grabbing headlines with larger semantic sizes in a real-world web shopping scenario. The results show that multi-modal LLMs are more emotionally engaged in decision-making, but this also introduces potential biases, such as the risk of manipulation through clickbait headlines. Ultimately, this study offers a novel perspective on how LLMs interpret and internalize language, from the smallest concrete objects to the most profound abstract concepts like love. The insights gained not only improve our understanding of LLMs but also provide new avenues for exploring the cognitive abilities that define human intelligence.
Problem
Research questions and friction points this paper is trying to address.
Investigates semantic size understanding in LLMs and humans.
Explores multi-modal training's role in human-like cognition.
Examines LLMs' emotional engagement and bias in decision-making.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Multi-modal training enhances LLM human-like cognition.
LLMs' internal representations align with human cognitive processes.
Multi-modal LLMs show emotional engagement in decision-making.
🔎 Similar Papers
No similar papers found.
