Building Trust in Virtual Immunohistochemistry: Automated Assessment of Image Quality

2025-11-06

AI Summary

Existing image fidelity metrics (e.g., FID, PSNR, SSIM) inadequately reflect pathological accuracy in virtual immunohistochemistry (IHC) staining. To address this, we propose the first annotation-free, pixel-level framework for assessing staining accuracy: it employs color deconvolution to generate diaminobenzidine (DAB)-like brown-stain masks and quantifies spatial consistency using segmentation metrics (Dice, IoU). Crucially, we extend evaluation to the whole-slide level—revealing critical limitations of patch-based assessment. Experiments across state-of-the-art methods—including PyramidPix2Pix, AdaptiveNCE, diffusion models, and GANs—demonstrate that paired image translation models significantly outperform unpaired ones in staining accuracy. Moreover, whole-slide analysis uncovers systematic performance degradation undetected by patch-level metrics. This framework establishes a reproducible, pathology-relevant quality benchmark for clinical translation of virtual IHC.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Calibration & Uncertainty Quantification
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Deep learning models can generate virtual immunohistochemistry (IHC) stains from hematoxylin and eosin (H&E) images, offering a scalable and low-cost alternative to laboratory IHC. However, reliable evaluation of image quality remains a challenge as current texture- and distribution-based metrics quantify image fidelity rather than the accuracy of IHC staining. Here, we introduce an automated and accuracy grounded framework to determine image quality across sixteen paired or unpaired image translation models. Using color deconvolution, we generate masks of pixels stained brown (i.e., IHC-positive) as predicted by each virtual IHC model. We use the segmented masks of real and virtual IHC to compute stain accuracy metrics (Dice, IoU, Hausdorff distance) that directly quantify correct pixel - level labeling without needing expert manual annotations. Our results demonstrate that conventional image fidelity metrics, including Frechet Inception Distance (FID), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM), correlate poorly with stain accuracy and pathologist assessment. Paired models such as PyramidPix2Pix and AdaptiveNCE achieve the highest stain accuracy, whereas unpaired diffusion- and GAN-based models are less reliable in providing accurate IHC positive pixel labels. Moreover, whole-slide images (WSI) reveal performance declines that are invisible in patch-based evaluations, emphasizing the need for WSI-level benchmarks. Together, this framework defines a reproducible approach for assessing the quality of virtual IHC models, a critical step to accelerate translation towards routine use by pathologists.

Problem

• Automated evaluation of virtual immunohistochemistry image quality
• Quantifying IHC stain accuracy without manual expert annotations
• Addressing poor correlation between conventional metrics and pathologist assessment

Innovation

• Automated framework assesses virtual IHC image quality
• Uses color deconvolution to generate IHC-positive pixel masks
• Computes stain accuracy metrics without expert annotations