Characterizing the Role of Similarity in the Property Inferences of Language Models

2024-10-29

arXiv.org

AI Summary

This study investigates the cognitive mechanisms underlying attribute inheritance in language models—e.g., inferring that “sparrow” possesses a novel attribute known for “bird”: whether such inference relies on explicit taxonomic knowledge or implicit representational similarity. Combining behavioral experiments with causal representational analysis—including interventional probing and quantification of similarity spaces—we demonstrate, for the first time, that these two mechanisms are not mutually exclusive but synergistic: robust attribute projection occurs only when both category membership is unambiguous and latent representations exhibit high similarity. This reveals a dual foundation of conceptual structure in current large language models, offering new empirical evidence for understanding their analogical reasoning and knowledge generalization capabilities. Our findings challenge prevailing theoretical frameworks that treat taxonomic knowledge and distributional representation as opposing paradigms, instead supporting an integrated account of semantic cognition in foundation models.

Technology Category

• Cognitive Modeling & Cognitive Systems: Analogy
• Knowledge Representation and Reasoning: Common-Sense Reasoning
• Natural Language Processing: (Large) Language Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Property inheritance -- a phenomenon where novel properties are projected from higher level categories (e.g., birds) to lower level ones (e.g., sparrows) -- provides a unique window into how humans organize and deploy conceptual knowledge. It is debated whether this ability arises due to explicitly stored taxonomic knowledge vs. simple computations of similarity between mental representations. How are these mechanistic hypotheses manifested in contemporary language models? In this work, we investigate how LMs perform property inheritance with behavioral and causal representational analysis experiments. We find that taxonomy and categorical similarities are not mutually exclusive in LMs' property inheritance behavior. That is, LMs are more likely to project novel properties from one category to the other when they are taxonomically related and at the same time, highly similar. Our findings provide insight into the conceptual structure of language models and may suggest new psycholinguistic experiments for human subjects.

Problem

• Investigate property inheritance in language models
• Explore role of taxonomy and similarity in LMs
• Analyze conceptual structure of language models

Innovation

• Behavioral and causal representational analysis experiments
• Investigating property inheritance in language models
• Combining taxonomy and categorical similarity analysis