Cross-channel Perception Learning for H&E-to-IHC Virtual Staining

2025-06-09

AI Summary

Existing H&E-to-IHC virtual staining methods neglect cross-channel structural correlations between nuclear (hematoxylin) and membranous (DAB) components. To address this, we propose a Cross-Channel Perception Learning (CCPL) framework that explicitly models their statistical and geometric dependencies in feature space—via dual-channel feature alignment, knowledge distillation loss, and optical density distribution consistency regularization. Leveraging the Gigapath tile encoder for multi-scale tissue representation, CCPL integrates channel decomposition and optical density analysis to generate high-fidelity HER2 IHC images. Experiments demonstrate state-of-the-art performance across quantitative metrics (PSNR, SSIM, PCC, FID) and expert pathological evaluation, with significant improvements in nuclear-membrane localization accuracy and staining intensity realism. This advances automated HER2 scoring by providing more biologically plausible and diagnostically reliable virtual stains.

Technology Category

• Computer Vision: Medical and Biological Imaging
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

With the rapid development of digital pathology, virtual staining has become a key technology in multimedia medical information systems, offering new possibilities for the analysis and diagnosis of pathological images. However, existing H&E-to-IHC studies often overlook the cross-channel correlations between cell nuclei and cell membranes. To address this issue, we propose a novel Cross-Channel Perception Learning (CCPL) strategy. Specifically, CCPL first decomposes HER2 immunohistochemical staining into Hematoxylin and DAB staining channels, corresponding to cell nuclei and cell membranes, respectively. Using the pathology foundation model Gigapath's Tile Encoder, CCPL extracts dual-channel features from both the generated and real images and measures cross-channel correlations between nuclei and membranes. The features of the generated and real stained images, obtained through the Tile Encoder, are also used to calculate feature distillation loss, enhancing the model's feature extraction capabilities without increasing the inference burden. Additionally, CCPL performs statistical analysis on the focal optical density maps of both single channels to ensure consistency in staining distribution and intensity. Experimental results, based on quantitative metrics such as PSNR, SSIM, PCC, and FID, along with professional evaluations from pathologists, demonstrate that CCPL effectively preserves pathological features, generates high-quality virtual stained images, and provides robust support for automated pathological diagnosis using multimedia medical data.

Problem

• Overcoming cross-channel correlation neglect in H&E-to-IHC virtual staining
• Enhancing feature extraction without increasing inference burden
• Ensuring staining distribution and intensity consistency in virtual images

Innovation

• Decomposes HER2 staining into dual channels
• Measures cross-channel nuclei-membrane correlations
• Ensures staining consistency via optical density