PCoA: A New Benchmark for Medical Aspect-Based Summarization With Phrase-Level Context Attribution

2026-01-06

arXiv.org

AI Summary

This work addresses the lack of fine-grained contextual provenance in existing medical summarization systems, which hinders accuracy verification. To this end, the authors construct PCoA, the first expert-annotated benchmark for aspect-level medical summarization, enabling precise alignment among summaries, supporting sentences, and contributive phrases. They further introduce a novel phrase-level contextual attribution mechanism and propose a decoupled evaluation framework that separately assesses the quality of summaries, citations, and contributive phrases. Experimental results demonstrate that the PCoA dataset exhibits high quality and inter-annotator consistency. Moreover, generating summaries after first identifying relevant sentences and key phrases significantly improves performance, establishing a reliable benchmark and a new paradigm for building trustworthy medical summarization systems.

Technology Category

• Natural Language Processing: Summarization
• Data Mining & Knowledge Management: Data Visualization & Summarization
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Verifying system-generated summaries remains challenging, as effective verification requires precise attribution to the source context, which is especially crucial in high-stakes medical domains. To address this challenge, we introduce PCoA, an expert-annotated benchmark for medical aspect-based summarization with phrase-level context attribution. PCoA aligns each aspect-based summary with its supporting contextual sentences and contributory phrases within them. We further propose a fine-grained, decoupled evaluation framework that independently assesses the quality of generated summaries, citations, and contributory phrases. Through extensive experiments, we validate the quality and consistency of the PCoA dataset and benchmark several large language models on the proposed task. Experimental results demonstrate that PCoA provides a reliable benchmark for evaluating system-generated summaries with phrase-level context attribution. Furthermore, comparative experiments show that explicitly identifying relevant sentences and contributory phrases before summarization can improve overall quality. The data and code are available at https://github.com/chubohao/PCoA.

Problem

• medical summarization
• context attribution
• phrase-level alignment
• aspect-based summarization
• summary verification

Innovation

• phrase-level attribution
• medical summarization
• aspect-based summarization
• context citation
• fine-grained evaluation