iSight: Towards expert-AI co-assessment for improved immunohistochemistry staining interpretation

2026-02-03

AI Summary

This study addresses the high subjectivity and poor inter-observer consistency in immunohistochemistry (IHC) staining interpretation, compounded by the limited applicability of existing AI models. To overcome these challenges, the authors construct HPA10M, a large-scale IHC dataset with ten million expert annotations, and propose iSight, a novel multi-task learning framework that introduces, for the first time, an expert–AI collaborative assessment paradigm. iSight integrates whole-slide images with tissue metadata and employs a token-level attention mechanism to jointly predict staining intensity, localization, cell count, tissue type, and malignancy status. On a held-out test set, iSight achieves accuracies of 85.5%, 76.6%, and 75.7% on localization, intensity, and quantity tasks, respectively, significantly outperforming fine-tuned baselines. User studies demonstrate that iSight enhances pathologists’ diagnostic accuracy and improves inter-rater agreement, raising Cohen’s κ from 0.63 to 0.70.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Immunohistochemistry (IHC) provides information on protein expression in tissue sections and is commonly used to support pathology diagnosis and disease triage. While AI models for H\&E-stained slides show promise, their applicability to IHC is limited due to domain-specific variations. Here we introduce HPA10M, a dataset that contains 10,495,672 IHC images from the Human Protein Atlas with comprehensive metadata included, and encompasses 45 normal tissue types and 20 major cancer types. Based on HPA10M, we trained iSight, a multi-task learning framework for automated IHC staining assessment. iSight combines visual features from whole-slide images with tissue metadata through a token-level attention mechanism, simultaneously predicting staining intensity, location, quantity, tissue type, and malignancy status. On held-out data, iSight achieved 85.5\% accuracy for location, 76.6\% for intensity, and 75.7\% for quantity, outperforming fine-tuned foundation models (PLIP, CONCH) by 2.5--10.2\%. In addition, iSight demonstrates well-calibrated predictions with expected calibration errors of 0.0150-0.0408. Furthermore, in a user study with eight pathologists evaluating 200 images from two datasets, iSight outperformed initial pathologist assessments on the held-out HPA dataset (79\% vs 68\% for location, 70\% vs 57\% for intensity, 68\% vs 52\% for quantity). Inter-pathologist agreement also improved after AI assistance in both held-out HPA (Cohen's $\kappa$ increased from 0.63 to 0.70) and Stanford TMAD datasets (from 0.74 to 0.76), suggesting expert--AI co-assessment can improve IHC interpretation. This work establishes a foundation for AI systems that can improve IHC diagnostic accuracy and highlights the potential for integrating iSight into clinical workflows to enhance the consistency and reliability of IHC assessment.

Problem

• Immunohistochemistry
• AI-assisted diagnosis
• staining interpretation
• pathologist agreement
• domain-specific variation

Innovation

• immunohistochemistry
• multi-task learning
• token-level attention
• expert-AI co-assessment
• HPA10M dataset