POLARIS: Guiding Small Models to Write Long Stories

2026-06-02

AI Summary

This work addresses the challenge that small-scale open-source language models struggle to simultaneously maintain generation length and content quality in long-form creative writing, particularly suffering significant performance degradation beyond their training sequence length. To overcome this limitation, the authors propose POLARIS-9B, a framework integrating GRPO reinforcement learning, an LLM-as-a-judge structured reward mechanism, and a human reference injection (HRI) strategy to enhance length generalization and narrative quality with minimal computational overhead. Built upon the Qwen3.5-9B base model and trained on 1.4K high-quality stories, POLARIS-9B matches or surpasses larger open-source models across five benchmarks. Blind evaluations demonstrate its substantial improvement over the base model and parity with Qwen3.5-27B, while preserving high-quality output and instruction adherence up to three times the original training length.

Abstract

Small open-weight models struggle at long-form creative writing: their generated stories either fall far short of the requested length, or their quality significantly degrades as length increases, especially when compared to frontier models. We present POLARIS (Policy Optimization with LLM-as-a-judge rewards and Anchored-Reference Injection for Storywriting), a lower-compute GRPO recipe with two key ingredients: a frontier LLM judge with a structured Story Quality rubric as the online reward, and human-reference injection (HRI), where a teacher-forced human-written story serves as a high-reward anchor within each GRPO group. By applying our training recipe to Qwen3.5-9B, using a dataset of approximately 1.4K prompt-story pairs derived from 100 short-story anthologies and 4 A100 GPUs, we obtain POLARIS-9B. Across five benchmarks spanning in-distribution and out-of-distribution prompts and rubrics, POLARIS-9B is competitive with much larger open-weight models while following length instructions more closely. A blinded human evaluation confirms that POLARIS-9B is preferred to the base Qwen3.5-9B and on par with Qwen3.5-27B. Despite training only on stories up to 4k words, POLARIS-9B preserves quality on prompts requesting stories up to 3 times the training length, a regime where most open-weight models degrade substantially in quality, length adherence, or both. More broadly, our results suggest that length generalization is a meaningful stress test for creative-writing models and a useful lens for distinguishing otherwise close models.

Problem

• long-form story generation
• length generalization
• small language models
• creative writing
• quality degradation

Innovation

• POLARIS
• length generalization
• human-reference injection
• LLM-as-a-judge
• GRPO