Technology Category
Computer Vision: Large Vision ModelsMachine Learning: Representation LearningNatural Language Processing: (Large) Language Models
Causality $\neq$ Invariance: Function and Concept Vectors in LLMs
🤖 AI Summary
This study investigates whether large language models form abstract conceptual representations in an input-format-invariant manner and clarifies the relationship between such representations and those underlying in-context learning (ICL). To this end, the authors introduce “concept vectors” (CVs) and conduct a systematic comparison using function vectors (FVs), representational similarity analysis (RSA), attention head composition, and cross-format and cross-lingual intervention experiments. The findings reveal that while FVs excel in in-distribution tasks, CVs demonstrate superior generalization across out-of-distribution problem types and languages. This indicates that models indeed encode abstract conceptual representations, yet these representations are fundamentally distinct from the mechanisms that drive ICL.
Technology Category
Application Category
📝 Abstract
Do large language models (LLMs) represent concepts abstractly, i.e., independent of input format? We revisit Function Vectors (FVs), compact representations of in-context learning (ICL) tasks that causally drive task performance. Across multiple LLMs, we show that FVs are not fully invariant: FVs are nearly orthogonal when extracted from different input formats (e.g., open-ended vs. multiple-choice), even if both target the same concept. We identify Concept Vectors (CVs), which carry more stable concept representations. Like FVs, CVs are composed of attention head outputs; however, unlike FVs, the constituent heads are selected using Representational Similarity Analysis (RSA) based on whether they encode concepts consistently across input formats. While these heads emerge in similar layers to FV-related heads, the two sets are largely distinct, suggesting different underlying mechanisms. Steering experiments reveal that FVs excel in-distribution, when extraction and application formats match (e.g., both open-ended in English), while CVs generalize better out-of-distribution across both question types (open-ended vs. multiple-choice) and languages. Our results show that LLMs do contain abstract concept representations, but these differ from those that drive ICL performance.
Problem
Research questions and friction points this paper is trying to address.
causality
invariance
concept representation
large language models
input format
Innovation
Methods, ideas, or system contributions that make the work stand out.
Concept Vectors
Function Vectors
Representational Similarity Analysis
In-context Learning
Format Invariance
🔎 Similar Papers
No similar papers found.
