Problem
Research questions and friction points this paper is trying to address.
Consolidation
Multi-Reference Training
Peer Reviews
Natural Language Generation
To Consolidate or not to Consolidate? Evaluating the Impact of Consolidation in Multi-Reference Training using Peer Reviews
๐ค AI Summary
็ ็ฉถ้่ฟๅผๅ
ฅMERC-36Kๆฐๆฎ้๏ผ่ฏไผฐไบๅจๅคๅ่่ฎญ็ปไธญๆดๅๅ่็้่ฆๆง๏ผๅนถ่ฏๆไบไฝฟ็จๆดๅๅ็ๅ่่ตๆ่ฎญ็ปๆจกๅ่ฝๆพ่ๆ้ซๆง่ฝใ
๐ Abstract
Natural language generation (NLG) tasks span the spectrum of conditional entropy, ranging from highly constrained machine translation to open-ended dialogue generation. Structured tasks like automated peer-review generation occupy the intermediate region, where a single input admits multiple valid, overlapping outputs. In this work, we demonstrate that traditional single- and multi-reference training paradigms are suboptimal for these intermediary tasks. We provide empirical evidence that consolidating diverse references into a unified training signal is crucial for developing effective systems. To facilitate this, we introduce MERC-36K, a large-scale corpus of over 36,000 papers paired with original and consolidated peer reviews. Using this dataset, we train specific architectures to isolate the impact of different reference paradigms and benchmark against existing state-of-the-art systems. Through extensive automatic and human evaluation, we demonstrate that models trained on consolidated references significantly outperform those trained on unconsolidated references. Dataset and code will be released upon acceptance.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Consolidated References
Natural Language Generation
Peer Review Generation
MERC-36K
๐ Similar Papers
No similar papers found.
