🤖 AI Summary
This paper investigates whether large language models (LLMs) possess genuine semantic understanding at lexical and sentential levels, focusing on core semantic phenomena—namely, reference and propositional attitude. Method: It introduces, for the first time, the Frege–Russell tradition of formal semantics to construct an integrated theoretical framework bridging philosophical semantics and computational analysis; combining Transformer-based modeling, semantic probing, Concept Activation Vectors (CAVs), and formal semantic modeling to empirically examine the internal semantic structure of LLMs’ linguistic representations. Contribution/Results: Results indicate that while LLMs exhibit semantically plausible behavior in specific tasks, they lack stable referential mechanisms and robust propositional attitude representations. Their “understanding” is fundamentally statistical association rather than meaning-based comprehension. The study establishes a cross-disciplinary methodology and theoretical criteria for evaluating semantic competence in AI systems.
📝 Abstract
Large Language Models (LLMs) such as ChatGPT demonstrated the potential to replicate human language abilities through technology, ranging from text generation to engaging in conversations. However, it remains controversial to what extent these systems truly understand language. We examine this issue by narrowing the question down to the semantics of LLMs at the word and sentence level. By examining the inner workings of LLMs and their generated representation of language and by drawing on classical semantic theories by Frege and Russell, we get a more nuanced picture of the potential semantic capabilities of LLMs.