Uni-Hema: Unified Model for Digital Hematopathology

2025-11-17

AI Summary

Digital hematopathology demands cell-level analysis across diverse disease categories—malignant, infectious, and non-malignant—yet existing single-task or unimodal models lack the capacity for cross-disease, multi-granularity unified inference. To address this, we propose Hema-Former, the first unified multi-task, multi-modal framework specifically designed for hematopathology. It jointly performs cell detection, segmentation, classification, morphological prediction, and cross-disease semantic reasoning, enabling pixel-to-semantic vision–language co-modeling. Integrating masked language modeling and visual question answering, Hema-Former is trained on 46 publicly available datasets and ensures single-cell-level interpretability. Experiments demonstrate that it matches or surpasses state-of-the-art specialized single-task models across all evaluated tasks. Notably, it achieves the first end-to-end, full-spectrum multi-task analysis covering all major hematologic diseases, thereby establishing a new paradigm for multi-task modeling in digital hematopathology.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Digital hematopathology requires cell-level analysis across diverse disease categories, including malignant disorders (e.g., leukemia), infectious conditions (e.g., malaria), and non-malignant red blood cell disorders (e.g., sickle cell disease). Whether single-task, vision-language, WSI-optimized, or single-cell hematology models, these approaches share a key limitation, they cannot provide unified, multi-task, multi-modal reasoning across the complexities of digital hematopathology. To overcome these limitations, we propose Uni-Hema, a multi-task, unified model for digital hematopathology integrating detection, classification, segmentation, morphology prediction, and reasoning across multiple diseases. Uni-Hema leverages 46 publicly available datasets, encompassing over 700K images and 21K question-answer pairs, and is built upon Hema-Former, a multimodal module that bridges visual and textual representations at the hierarchy level for the different tasks (detection, classification, segmentation, morphology, mask language modeling and visual question answer) at different granularity. Extensive experiments demonstrate that Uni-Hema achieves comparable or superior performance to train on a single-task and single dataset models, across diverse hematological tasks, while providing interpretable, morphologically relevant insights at the single-cell level. Our framework establishes a new standard for multi-task and multi-modal digital hematopathology. The code will be made publicly available.

Problem

• Unifying multi-task analysis across diverse hematological diseases and modalities
• Overcoming limitations of single-task models in digital hematopathology diagnostics
• Integrating detection, classification, segmentation and reasoning for blood disorders

Innovation

• Multi-task unified model for digital hematopathology analysis
• Integrates detection classification segmentation across multiple diseases
• Uses multimodal transformer bridging visual textual representations