CellGenNet: A Knowledge-Distilled Framework for Robust Cell Segmentation in Cancer Tissues

2025-11-19

AI Summary

Nuclear segmentation in whole-slide images (WSIs) suffers from poor robustness due to staining variability, imaging artifacts, and morphological heterogeneity across tissue types. Method: We propose a knowledge distillation–based cross-tissue semi-supervised segmentation framework. It employs a teacher-student collaborative architecture to generate high-quality soft pseudo-labels, jointly optimizes binary cross-entropy and Tversky loss, and incorporates inter-layer dropout and consistency regularization to mitigate class imbalance, enhance feature stability, and preserve sparse nuclear structures. Contribution/Results: Evaluated on multiple cancer WSI datasets, our method achieves significant improvements over both fully supervised and state-of-the-art semi-supervised approaches using only minimal labeled data. It attains higher Dice scores and superior generalizability across diverse tissue types, establishing a scalable and reproducible paradigm for digital pathology analysis.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Semi-Supervised Learning
• Search and Optimization: Distributed Search

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Accurate nuclei segmentation in microscopy whole slide images (WSIs) remains challenging due to variability in staining, imaging conditions, and tissue morphology. We propose CellGenNet, a knowledge distillation framework for robust cross-tissue cell segmentation under limited supervision. CellGenNet adopts a student-teacher architecture, where a capacity teacher is trained on sparse annotations and generates soft pseudo-labels for unlabeled regions. The student is optimized using a joint objective that integrates ground-truth labels, teacher-derived probabilistic targets, and a hybrid loss function combining binary cross-entropy and Tversky loss, enabling asymmetric penalties to mitigate class imbalance and better preserve minority nuclear structures. Consistency regularization and layerwise dropout further stabilize feature representations and promote reliable feature transfer. Experiments across diverse cancer tissue WSIs show that CellGenNet improves segmentation accuracy and generalization over supervised and semi-supervised baselines, supporting scalable and reproducible histopathology analysis.

Problem

• Robust cell segmentation in cancer tissues with staining variability
• Knowledge distillation framework for limited supervision scenarios
• Mitigating class imbalance to preserve minority nuclear structures

Innovation

• Knowledge distillation framework with student-teacher architecture
• Joint objective integrating ground-truth and probabilistic targets
• Hybrid loss combining cross-entropy and Tversky loss