Multilingual Sentiment Aware Text Summarization A Reinforcement Learning Approach for Consistency Maintenance

2026-06-07

AI Summary

This study addresses the issue of sentiment drift in text summarization induced by reinforcement learning from human feedback (RLHF)—a phenomenon wherein generated summaries tend toward emotional neutrality and fail to preserve the source text’s affective tone. Through multilingual experiments, the work reveals for the first time the prevalence of this drift across languages and identifies KL regularization as a primary cause of sentiment suppression. To mitigate this, the authors propose a sentiment-aware KL regularization method that effectively alleviates sentiment drift across eight languages and multiple datasets while maintaining factual consistency and summary quality. Furthermore, they develop a policy attribution framework to quantitatively assess the contribution of individual components to sentiment shifts, offering a novel approach to controllable alignment in RLHF.

Abstract

Reinforcement Learning from Human Feedback (RLHF) has significantly improved the quality and fluency of large language models in text summarization. However, its impact on affective properties remains insufficiently understood. In this work, we study sentiment drift, a systematic shift toward neutral sentiment in RLHF-based summarization outputs compared to source texts. We conduct extensive experiments across multiple datasets, model architectures, and eight languages to analyze how alignment objectives influence sentiment preservation. Our results show that sentiment drift is a consistent phenomenon that becomes stronger with increased KL regularization strength, indicating a trade-off between alignment stability and affective fidelity. To explain this behavior, we introduce a Policy Attribution framework that decomposes the RLHF objective and quantifies the contribution of its components. Our analysis reveals that KL regularization is the primary driver of sentiment suppression across all settings. Based on these findings, we propose a sentiment-aware modification of the KL regularization term, which selectively reduces constraints on sentiment-bearing tokens. Empirical results demonstrate that this approach mitigates sentiment drift while maintaining summarization quality. Overall, our findings highlight a fundamental limitation of current alignment methods: while they improve factual consistency and safety, they may unintentionally suppress emotional expressiveness. This motivates the development of alignment strategies that explicitly account for affective preservation.

Problem

• sentiment drift
• reinforcement learning
• text summarization
• affective preservation
• KL regularization

Innovation

• sentiment drift
• reinforcement learning from human feedback
• KL regularization
• affective preservation
• multilingual summarization