A Multi-modal Agentic Co-pilot for Evidence Grounded Computational Pathology

2026-06-06

AI Summary

This study addresses the limited integration of artificial intelligence in pathology with evidence-based medicine, particularly the prevailing reliance on unimodal text lacking traceable evidence. The authors introduce, for the first time, a hierarchical evidence framework into computational pathology, constructing the most comprehensive multimodal pathology evidence corpus and a hypergraph-based knowledge engine to date. They propose a multi-agent collaborative reasoning framework that enables interpretable and traceable diagnostic inference from textual queries to specific regions in whole-slide images (WSIs). By integrating multimodal large language models, evidence retrieval, and WSI understanding, the method significantly outperforms existing approaches across more than 200,000 real-world cases. User studies demonstrate that the system effectively enhances pathologists’ diagnostic accuracy and decision confidence.

Abstract

Pathology is the cornerstone of modern medicine, where accurate decision-making relies heavily on evidence-based practices. While artificial intelligence (AI) has the potential to transform clinical workflows, the intersection of AI and evidence-based medicine remains under-explored, with primitive attempts restricted to text-only general medicine. In this work, we present PathPocket, a multimodal AI agentic co-pilot designed specifically for evidence grounded pathology. We construct the most comprehensive pathology evidence corpus to date, encompassing approximately 110,472 public and authorized documents structured across a rigorous hierarchy of evidence from clinical guideline to expert opinion. From this meticulously graded foundation, we build a large-scale multimodal pathology hypergraph containing over 4.55 million entities and 7.10 million relations. Serving as a robust knowledge engine, this hypergraph provides traceable evidence for a collaborative multi-agent reasoning framework integrating input understanding, evidence retrieval, filtering, and diagnosis generation. This enables PathPocket to seamlessly resolve a wide spectrum of clinical tasks, ranging from text-only queries to complex multimodal diagnostics involving region-of-interest (ROI) and gigapixel whole-slide images (WSIs). We rigorously evaluate the system on a multidimensional benchmark of over 200,000 real-world cases, where it significantly outperforms existing state-of-the-arts. Crucially, extensive user studies demonstrate that PathPocket substantially improves the diagnostic accuracy and confidence of pathologists. By directly grounding pathology interpretations in verifiable literature, PathPocket offers a practical and scalable solution for the future of evidence grounded computational pathology.

Problem

• evidence-based medicine
• computational pathology
• multimodal AI
• whole-slide images
• clinical decision-making

Innovation

• multimodal agentic co-pilot
• evidence-grounded pathology
• pathology hypergraph
• whole-slide image (WSI) analysis
• multi-agent reasoning