Improving Diversity in LLM Short Story Generation

2026-10-05

AI Summary

This study addresses the homogenization of content generated by large language models by proposing DivLM, a framework designed to enhance narrative diversity in short stories across dimensions such as genre and style. Methodologically, DivLM employs continual pre-training combined with weight residual techniques to preserve instruction-following capabilities. Furthermore, it introduces reinforcement learning to optimize a novel composite reward function over narrative dimensions, achieving a synergistic balance between diversity and generation quality. Experimental results demonstrate that the proposed approach improves diversity metrics by an average of over 9% while maintaining high-quality outputs and human-like writing styles.

Abstract

Large language models (LLMs) can generate accurate responses, but these are void of diversity. We attempt to address this for the task of creative short story generation. Drawing on established writing conventions and known LLM limitations, we target variation in genre, tone, style, and named entities. To promote diversity across these dimensions, we introduce DivLM, an LLM post-training framework consisting of two phases. First, we perform continued pre-training on a creative writing corpus and restore instruction-following capabilities using weight residuals. We then apply reinforcement learning with a custom, composite reward function that jointly maximizes diversity across the targeted narrative dimensions while maintaining response quality. Our empirical results on two LLM families show that DivLM increases diversity metrics by more than 9% on average compared to alternative approaches, while preserving instruction following, overall response quality, and similarity to human outputs.

Problem

• Large Language Models
• Diversity
• Short Story Generation
• Creative Writing

Innovation

• Diversity
• Large Language Models
• Post-training Framework
• Weight Residuals
• Reinforcement Learning