Back to the Future: The Role of Past and Future Context Predictability in Incremental Language Production

2026-04-11

AI Summary

This study investigates how both past and future contextual predictability influence lexical form selection and encoding in online language production, focusing on the “backward constraint” effect—where future context retroactively constrains current word choice—and its underlying cognitive planning mechanisms. Method: We propose a novel information-theoretic measure of predictability incorporating bidirectional context (forward + backward), applied to natural corpora via enhanced language models; we further employ generative modeling to integrate lexical, contextual, and communicative factors, systematically identifying alternative speech error types and their causes. Contribution/Results: The backward predictability metric significantly predicts word length and error distributions; distinct error patterns reflect speakers’ real-time trade-offs among form, meaning, and future contextual information during incremental planning. This work provides the first computational formalization and empirical validation of backward predictability, establishing its functional role in sentence production.

Technology Category

Natural Language Processing: Lexical Semantics and MorphologyPlanning, Routing, and Scheduling: Planning with Language ModelsCognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

Economics, Online Markets and Human Computation: Cost models of using LLMs in production systemsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Contextual predictability shapes both the form and choice of words in online language production. The effects of the predictability of a word given its previous context are generally well-understood in both production and comprehension, but studies of naturalistic production have also revealed a poorly-understood backward predictability effect of a word given its future context, which may be related to future planning. Here, in two studies of naturalistic speech corpora, we investigate backward predictability effects using improved measures and more powerful language models, introducing a new principled and conceptually motivated information-theoretic predictability measure that integrates predictability from both the future and the past context. Our first study revisits classic predictability effects on word duration. Our second study investigates substitution errors within a generative framework that independently models the effects of lexical, contextual, and communicative factors on word choice, while predicting the actual words that surface as speech errors. We find that our proposed conceptually-motivated alternative to backward predictability yields qualitatively similar effects across both studies. Through a fine-grained analysis of substitution errors, we further show that different kinds of errors are suggestive of how speakers prioritize form, meaning, and context-based information during lexical planning. Together, these findings illuminate the functional roles of past and future context in how speakers encode and choose words, offering a bridge between contextual predictability effects and the mechanisms of sentence planning.

Problem

• Investigating backward predictability effects of future context on language production
• Developing an information-theoretic measure integrating past and future predictability
• Analyzing substitution errors to understand lexical planning mechanisms

Innovation

• Integrated past and future context predictability measure
• Generative framework modeling lexical and contextual factors
• Fine-grained analysis of substitution errors in speech