Script Choice in LLMs: Evidence for Late-Layer Commitment

2026-09-23

AI Summary

This study investigates the inter-layer distribution mechanisms of script knowledge and the formation of output script commitment in large language models. Using logistic regression probes and LogitLens analysis, we reveal a two-stage asymmetry in script processing: input and instruction scripts are encoded in early layers, intermediate layers default to Latin characters, and actual output script commitment emerges exclusively in the final layer. Our findings demonstrate that this commitment capacity correlates positively with model depth, while smaller models exhibit comparatively weaker performance. These results establish the critical role of model depth in multilingual generation, offering new perspectives for designing deeper and more inclusive multilingual architectures.

Abstract

In this paper, we investigate how script knowledge is distributed across the layers of LLMs using two complementary interpretability methods: logistic regression probing and logit-lens analysis. Our probing experiments reveal a clear asymmetry: both the input script and the instructed output script are encoded in the earliest layers of the network, while, in contrast, commitment to the actual output script emerges only in the final layers, with the model's intermediate representations defaulting to Latin throughout most of the layers. This two-stage process is confirmed by logit-lens analyses, which show that script commitment consistently occurs at the very last layers of the LLMs. Together with the weaker script-following performance observed in smaller models, these results form a converging body of evidence linking script commitment to model depth, with broader implications for the design of sufficiently deep, inclusive multilingual architectures.

Problem

• script choice
• large language models
• layer commitment
• multilingual
• interpretability

Innovation

• script commitment
• interpretability
• logit-lens analysis
• probing
• multilingual LLMs