Technology Category
Computer Vision: Multi-modal VisionMachine Learning: Deep Neural Architectures and Foundation ModelsPlanning, Routing, and Scheduling: Planning with Language Models
PLUTO-4: Frontier Pathology Foundation Models
🤖 AI Summary
To address the weak multi-task transferability and poor generalizability of existing histopathological image models, this work introduces the PLUTO-4 family of foundation models for pathology. Methodologically, we propose a dual-branch Vision Transformer architecture—comprising the lightweight PLUTO-4S and high-performance PLUTO-4G—incorporating FlexiViT for dynamic resolution adaptation and 2D Rotary Position Embedding (2D-RoPE). Leveraging the DINOv2 framework, we perform self-supervised pretraining on 550,000 multi-center whole-slide images. Our approach enables flexible scale inference from single-resolution training and significantly enhances robustness across institutions, disease types, and staining protocols. Experiments demonstrate that PLUTO-4G achieves an 11% accuracy gain in skin pathology diagnosis, while PLUTO-4S balances high throughput with stability. Both variants attain state-of-the-art performance across diverse benchmarks—including tile-level classification, segmentation, and slide-level diagnosis—thereby strongly supporting clinical translation.
Technology Category
Application Category
📝 Abstract
Foundation models trained on large-scale pathology image corpora have demonstrated strong transfer capabilities across diverse histopathology tasks. Building on this progress, we introduce PLUTO-4, our next generation of pathology foundation models that extend the Pathology-Universal Transformer (PLUTO) to frontier scale. We share two complementary Vision Transformer architectures in the PLUTO-4 family: a compact and efficient PLUTO-4S model optimized for multi-scale deployment using a FlexiViT setup with 2D-RoPE embeddings, and a frontier-scale PLUTO-4G model trained with a single patch size to maximize representation capacity and stability. Both models are pretrained using a self-supervised objective derived from DINOv2 on a large multi-institutional corpus containing 551,164 WSIs from 137,144 patients across over 50 institutions, spanning over 60 disease types and over 100 stains. Comprehensive evaluation across public and internal benchmarks demonstrates that PLUTO-4 achieves state-of-the-art performance on tasks requiring varying spatial and biological context, including tile classification, segmentation, and slide-level diagnosis. The compact PLUTO-4S provides high-throughput and robust performance for practical deployment, while PLUTO-4G establishes new performance frontiers across multiple pathology benchmarks, including an 11% improvement in dermatopathology diagnosis. These diverse improvements underscore PLUTO-4's potential to transform real-world applications as a backbone for translational research and diagnostic use cases.
Problem
Research questions and friction points this paper is trying to address.
Developing scalable pathology foundation models for diverse histopathology tasks
Creating efficient and high-capacity vision transformers for medical image analysis
Advancing diagnostic accuracy across multiple pathology benchmarks and applications
Innovation
Methods, ideas, or system contributions that make the work stand out.
PLUTO-4 introduces compact and frontier-scale Vision Transformer architectures
Models use self-supervised DINOv2 training on multi-institutional pathology corpus
FlexiViT setup with 2D-RoPE enables optimized multi-scale deployment
🔎 Similar Papers
No similar papers found.
