Policy-based Sentence Simplification: Replacing Parallel Corpora with LLM-as-a-Judge

2025-12-05

AI Summary

Sentence simplification requires adapting diverse strategies (e.g., lexical substitution or full-sentence rewriting), yet existing approaches rely heavily on parallel corpora or manual annotations, hindering fine-grained strategy control. This paper introduces the first parallel-corpus-free, strategy-driven simplification framework: it leverages large language models as judges (LLM-as-a-Judge) to generate high-quality, strategy-aligned training data via instruction-guided generation and feedback-based scoring; subsequently, a lightweight open-source model (e.g., Phi-3-mini-3.8B) is fine-tuned on this data. Experiments show that our method surpasses GPT-4o in lexical simplification and matches it in overall rewriting quality, with both automatic metrics and human evaluation confirming its effectiveness, generalizability, and robustness. The core contribution lies in decoupling strategy control from data construction—enabling zero-shot, multi-strategy, low-resource controllable simplification for the first time.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Sentence simplification aims to modify a sentence to make it easier to read and understand while preserving the meaning. Different applications require distinct simplification policies, such as replacing only complex words at the lexical level or rewriting the entire sentence while trading off details for simplicity. However, achieving such policy-driven control remains an open challenge. In this work, we introduce a simple yet powerful approach that leverages Large Language Model-as-a-Judge (LLM-as-a-Judge) to automatically construct policy-aligned training data, completely removing the need for costly human annotation or parallel corpora. Our method enables building simplification systems that adapt to diverse simplification policies. Remarkably, even small-scale open-source LLMs such as Phi-3-mini-3.8B surpass GPT-4o on lexical-oriented simplification, while achieving comparable performance on overall rewriting, as verified by both automatic metrics and human evaluations. The consistent improvements across model families and sizes demonstrate the robustness of our approach.

Problem

• Automates policy-aligned training data creation without human annotation
• Enables adaptable sentence simplification systems for diverse policies
• Achieves competitive performance with small-scale models on simplification tasks

Innovation

• LLM-as-a-Judge constructs policy-aligned training data
• Removes need for human annotation or parallel corpora
• Enables building adaptable simplification systems for diverse policies