When Evidence Conflicts: Reliability-aware Meta-review Generation

2026-09-20

AI Summary

本文针对同行评审中证据冲突和可靠性差异问题，提出一种基于证据可靠性聚合的元评审生成方法，有效识别并解决冲突，提升评审质量。

Abstract

Generating coherent meta-reviews from multiple peer reviews is challenging when reviewer evidence conflicts and varies in reliability. Existing approaches typically formulate meta-review generation as a multi-document summarization task and aggregate reviewer feedback uniformly, making it difficult to determine which opinions should be prioritized under disagreement. In this paper, we study meta-review generation through reliability-aware evidence aggregation. Our framework first extracts aspect-level opinions from peer reviews and identifies conflicting evidence within each aspect. It then estimates opinion-level support and review-level quality to measure evidence reliability. Based on these signals, the framework assigns reliability-aware weights to reviewer feedback, enabling the generator to prioritize better-supported arguments while preserving diverse perspectives. Experiments demonstrate that our method consistently improves meta-review generation over strong baselines on both automatic and human evaluations, with clear gains in conflict recognition and resolution under high-conflict review scenarios. The code and implementation details are publicly available at https://github.com/Wangxz729/reliability-aware-meta-review.

Problem

• conflict
• reliability
• meta-review generation
• peer reviews
• evidence aggregation

Innovation

• reliability-aware
• evidence aggregation
• conflict recognition
• meta-review generation
• opinion-level support