Transforming H&E images into IHC: A Variance-Penalized GAN for Precision Oncology

2025-06-23

AI Summary

To address the high cost, time consumption, and antibody dependency of HER2 immunohistochemistry (IHC) testing, this work proposes a deep learning–based H&E-to-IHC image translation method that synthesizes high-fidelity HER2 IHC images from routine hematoxylin–eosin (H&E)-stained slides. Methodologically, we design an enhanced pyramid pix2pix architecture incorporating a variance penalty term to mitigate mode collapse in GAN training, thereby improving structural fidelity and morphological diversity—particularly for complex HER2-positive (IHC 3+) patterns. Evaluated on the BCI dataset, our approach achieves superior performance over state-of-the-art methods in PSNR, SSIM, and FID metrics, with marked gains in translating HER2-positive cases. This advancement enhances diagnostic accessibility and scalability while demonstrating promising generalizability to other biomarkers.

Technology Category

Computer Vision: Medical and Biological ImagingMachine Learning: Deep Neural Architectures and Foundation ModelsNatural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphsEconomics, Online Markets and Human Computation: Data quality aspects of human-annotated datasetsSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The overexpression of the human epidermal growth factor receptor 2 (HER2) in breast cells is a key driver of HER2-positive breast cancer, a highly aggressive subtype requiring precise diagnosis and targeted therapy. Immunohistochemistry (IHC) is the standard technique for HER2 assessment but is costly, labor-intensive, and highly dependent on antibody selection. In contrast, hematoxylin and eosin (H&E) staining, a routine histopathological procedure, offers broader accessibility but lacks HER2 specificity. This study proposes an advanced deep learning-based image translation framework to generate highfidelity IHC images from H&E-stained tissue samples, enabling cost-effective and scalable HER2 assessment. By modifying the loss function of pyramid pix2pix, we mitigate mode collapse, a fundamental limitation in generative adversarial networks (GANs), and introduce a novel variance-based penalty that enforces structural diversity in generated images. Our model particularly excels in translating HER2-positive (IHC 3+) images, which have remained challenging for existing methods due to their complex morphological variations. Extensive evaluations on the BCI histopathological dataset demonstrate that our model surpasses state-of-the-art methods in terms of peak signal-tonoise ratio (PSNR), structural similarity index (SSIM), and Frechet Inception Distance (FID), particularly in accurately translating HER2-positive (IHC 3+) images. Beyond medical imaging, our model exhibits superior performance in general image-to-image translation tasks, showcasing its potential across multiple domains. This work marks a significant step toward AI-driven precision oncology, offering a reliable and efficient alternative to traditional HER2 diagnostics.

Problem

• Convert H&E images to IHC for HER2 assessment
• Overcome mode collapse in GANs for image translation
• Improve accuracy in translating HER2-positive (IHC 3+) images

Innovation

• Variance-penalized GAN for image diversity
• Modified pyramid pix2pix loss function
• Translates H&E to IHC for HER2 assessment