Beyond Score Prediction: LLM-Based Essay Scoring and Feedback Generation via Reinforcement Learning with Rubric Rewards

2026-07-21

AI Summary

This work addresses the lack of systematic optimization and quantifiable evaluation of feedback quality in large language models for automated essay scoring and feedback generation. The authors propose RLAES, a framework that jointly optimizes scoring and feedback generation via reinforcement learning. It introduces a rubric-based feedback evaluation (RFE) system grounded in 166 fine-grained binary criteria, an adaptive gated feedback optimization (AGFO) reward mechanism, and an adjacent contrastive reasoning (ACR) method for ordinal calibration. Experimental results on the ASAP benchmark demonstrate that RLAES achieves state-of-the-art scoring performance (QWK = 0.803) and generates feedback quality comparable to that of GPT-5.5, effectively mitigating the degradation of feedback commonly observed when optimizing solely for score accuracy.

Abstract

Large language models (LLMs) have been widely applied to automated essay scoring (AES) and automated feedback generation (AFG). However, existing studies rely primarily on prompt engineering or supervised fine-tuning, while systematic research on reinforcement learning (RL) post-training and automated evaluation of feedback quality remains limited. We propose RLAES, a unified LLM framework that jointly optimizes essay scoring and feedback generation through RL. To make feedback quality measurable, interpretable, and usable for training, we introduce Rubric-based Feedback Evaluation (RFE), an essay-grounded feedback evaluation framework comprising 166 fine-grained binary rubric items and an LLM-as-judge. Building on RFE, we propose Adaptive Gated Feedback Optimization (AGFO), which activates rubric-based feedback rewards on demand during RL, reducing evaluation overhead while improving feedback quality. We also propose Adjacent Contrastive Reasoning (ACR) to improve ordinal score calibration by explicitly contrasting adjacent score levels. Experimental results show that the RFE framework captures essay-feedback consistency, exhibits strong pairwise discriminative power, and closely aligns with expert preferences. On the ASAP benchmark, RLAES-AGFO achieves the best scoring performance among LLM-based methods (QWK = 0.803), while maintaining feedback quality comparable to GPT-5.5 and avoiding the feedback degradation observed under score-only RL. Code and datasets are publicly available at https://github.com/hellomuyi/RLAES.

Problem

• automated essay scoring
• automated feedback generation
• reinforcement learning
• feedback evaluation
• large language models

Innovation

• Reinforcement Learning
• Automated Essay Scoring
• Feedback Generation
• Rubric-based Evaluation
• Contrastive Reasoning