CraftAlign: Feature-Grounded Evaluation and Revision Guidance for AI Stories

2026-08-02

AI Summary

This work addresses the lack of naturalness in AI-generated stories, which often suffer from clichés, over-explanation, and stereotypical endings, and proposes a novel approach for globally coherent and diverse narrative optimization. The authors introduce a class-conditional energy-based model that explicitly captures stylistic differences between human and AI writing through a 304-dimensional narrative and style feature space. By integrating schema-valid structured perturbations with natural language guidance, the method enables targeted rewriting of a story’s overall structure, pacing of information disclosure, and ending. A feature estimator based on Qwen3.5-9B effectively identifies stylistic discrepancies, and the proposed framework significantly outperforms baseline methods in both automatic and human evaluations, offering an interpretable and controllable pathway toward high-quality narrative generation.

Abstract

Large language models can now generate fluent and complete stories, yet many outputs still feel formulaic and unnatural because of cliches, over-explanation, linear causal progression, and stereotyped endings, an immediately recognizable AI flavor. Existing detection and evaluation methods often stop at source labels or holistic scores, while revision methods typically target predefined issues through localized edits, limiting their ability to support multiple plausible revision strategies or guide story-wide changes in information release, causal organization, and ending treatment. We introduce CraftAlign, a framework that aligns AI stories with the craft of human storytelling by both assessing Human/AI writing patterns and providing revision guidance. CraftAlign comprises two learned modules and an inference-time guidance pipeline. A feature estimator built on Qwen3.5-9B predicts 304 explicit writing features spanning style and narrative. A class-conditional energy model scores the resulting feature configuration against Human and AI writing patterns, conditioning on the original writing prompt when available. At inference time, CraftAlign applies schema-valid structured perturbations, selects changes that move the feature configuration toward the Human writing pattern, and converts them into natural-language guidance for a separate editor to rewrite the full story. Experiments show that CraftAlign accurately distinguishes Human and AI writing patterns and that its guidance outperforms revision baselines across editors and in a human study.

Problem

• AI storytelling
• story evaluation
• revision guidance
• human-like narrative
• writing patterns

Innovation

• feature-grounded evaluation
• class-conditional energy model
• structured perturbation
• narrative revision guidance
• human-like storytelling