Large Language Models Preserve Semantic Isotopies in Story Continuations

2025-10-05

AI Summary

This study investigates large language models’ (LLMs) capacity to preserve semantic isotopies—coherent, recurring semantic threads—during story continuation. Using 10,000 ROCStories prompts, five state-of-the-art models (including GPT-4o) generated continuations, enabling the first systematic evaluation of semantic structural continuity across character, event, emotion, and theme dimensions. Methodologically, we integrate distributional semantic modeling with structured narrative analysis, proposing three interpretable quantitative metrics—coverage, density, and diffusion—to measure isotopic preservation. Results show that, under fixed output length constraints, all models maintain semantic isotopy robustly, exhibiting particularly strong consistency in core narrative elements. This work extends the empirical foundation of semantic coherence theory within generative AI and introduces a novel, fine-grained evaluation paradigm for controllable narrative generation.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Game Design — Procedural Content Generation & Storytelling

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

In this work, we explore the relevance of textual semantics to Large Language Models (LLMs), extending previous insights into the connection between distributional semantics and structural semantics. We investigate whether LLM-generated texts preserve semantic isotopies. We design a story continuation experiment using 10,000 ROCStories prompts completed by five LLMs. We first validate GPT-4o's ability to extract isotopies from a linguistic benchmark, then apply it to the generated stories. We then analyze structural (coverage, density, spread) and semantic properties of isotopies to assess how they are affected by completion. Results show that LLM completion within a given token horizon preserves semantic isotopies across multiple properties.

Problem

• Investigating whether LLMs preserve semantic isotopies in generated story continuations
• Analyzing structural and semantic properties of isotopies across multiple LLMs
• Validating GPT-4o's capability to extract isotopies from linguistic benchmarks

Innovation

• Validated GPT-4o for semantic isotopy extraction
• Designed story continuation experiment with five LLMs
• Analyzed structural and semantic isotopy preservation properties