Weakly Supervised Pathology-Informed Representation Learning for PET-Based Content Retrieval of Intra-Tumour Heterogeneity

2026-07-21

AI Summary

This study addresses the challenge of enhancing tumor heterogeneity-aware content retrieval using only PET images during inference. The authors propose a weakly supervised representation learning framework that leverages H&E histopathology as a weak supervisory signal during training to guide PET feature learning, while requiring only PET data at inference time. A teacher–student strategy is employed to generate both global and hotspot-conditioned embeddings that capture intra-tumoral heterogeneity. By innovatively integrating pathological weak supervision with PET hotspot modeling, the method significantly improves sensitivity to heterogeneity without compromising single-modality inference. Experiments demonstrate that the learned hotspot-conditioned representations outperform global PET features and conventional baselines, yielding substantial gains in retrieval performance (measured by mAP, nDCG, and MRR), with the derived prototype categories showing indirect alignment with histological characteristics.

Abstract

We propose a weakly supervised 18FFDG PET representation-learning framework for content based medical image retrieval, using H&E derived information during training while preserving PET-only inference. The proposed method was designed to use H&E derived information during training while maintaining PET only inference. A teacher student training strategy was used to learn the PET tumour derived voxel representations, from which global and hotspot conditioned embeddings were generated along with maps of intra tumour heterogeneity in our oesophegeal cancer test case. A progressive ablation strategy was used to evaluate the contribution of different supervision mechanisms. Retrieval performance was assessed across cross-validation folds using metrics including mean average precision, normalised discounted cumulative gain and mean reciprocal rank. Additional analyses evaluated ablation performance, hotspot faithfulness through perturbation/deletion experiments, prototype-specific PET uptake behaviour and indirect patient level concordance between learned PET prototype classes and selected histomic features. Progressive introduction of pathology informed supervision and hotspot modelling improved PET retrieval performance compared with global PET representations and conventional PET baselines. Across the ablation ladder, PET hotspot conditioned representations consistently provided stronger retrieval than global embeddings, indicating that focusing on informative tumour subregions improved sensitivity to intra tumour heterogeneity. Histopathology concordance further showed that the learned classes were not simply high uptake PET regions; instead, they demonstrated distinct heterogeneity in 18F FDG uptake.

Problem

• weakly supervised learning
• PET imaging
• intra-tumour heterogeneity
• content-based retrieval
• pathology-informed representation

Innovation

• weakly supervised learning
• pathology-informed representation
• hotspot-conditioned embedding
• intra-tumour heterogeneity
• teacher-student framework