On the Persistent Effects of Lexicality in Large Language Mod

2026-06-01

AI Summary

This study investigates how lexical overlap disproportionately dominates the representations of large language models, thereby degrading semantic expressiveness and impairing downstream task performance. Through adversarial semantic stress tests, information-theoretic analysis, representational similarity evaluation, and model editing experiments, the work systematically quantifies the persistent influence of lexical effects across varying model depths, architectures, and training objectives. The findings reveal that lexical signals permeate all layers—most prominently in intermediate layers, where a transitional degradation of both semantic and lexical information occurs simultaneously. Critically, such lexical interference substantially compromises the quality of semantic representations and induces performance biases in tasks such as summarization and model editing, highlighting the challenge of enhancing semantic robustness in current language models.

Abstract

Representations extracted from large language models (LLMs) play an important role in many downstream applications. However, the structure of these representations is often influenced by lexical overlap rather than semantic content. Our understanding of the relationship between this lexical influence and semantic content, and its implications for downstream tasks, remains limited. In this work, we investigate representations to quantify the effect of lexical overlap relative to semantic content. We consider several adversarial semantic stress tests and further connect our findings to the information theory perspective. We find that lexical influence extends across the depth of models, consistently across architectures, training regimes, and objective functions, including the models trained for semantic similarity. Moreover, we observe a mid-depth region in which both lexical and semantic signals degrade simultaneously, indicating a transitional regime where representations are poor for both surface form and meaning. We further demonstrate the effect of lexical influence on downstream uses of LLMs using summarization and model editing as a case study.

Problem

• lexicality
• semantic content
• large language models
• representations
• lexical overlap

Innovation

• lexicality
• semantic representation
• large language models
• adversarial stress tests
• information theory