Towards Segmenting the Invisible: An End-to-End Registration and Segmentation Framework for Weakly Supervised Tumour Analysis

2026-02-05

AI Summary

This work addresses the challenge of intraoperative liver tumor segmentation in CT, where tumors exhibit extremely low contrast against surrounding tissue, rendering them nearly invisible. In contrast, preoperative MRI clearly delineates lesions. The authors propose the first end-to-end framework for cross-modal registration and weakly supervised segmentation that operates under the extreme setting where pathological structures are entirely absent in the target modality (CT). By leveraging MRI-to-CT registration to generate pseudo-labels, the method enables segmentation of otherwise invisible tumors. It integrates MSCGUNet for multimodal registration and UNet for segmentation, explicitly revealing two core challenges: domain shift and feature absence. Experiments show a Dice score of 0.72 on the CHAOS dataset for healthy liver segmentation, but performance drops sharply to 0.16 on real clinical data containing tumors, highlighting the fundamental limitations of current weakly supervised approaches for truly invisible pathology segmentation.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Multimodal Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Security and Privacy: Data transparency and provenance

Abstract

Liver tumour ablation presents a significant clinical challenge: whilst tumours are clearly visible on pre-operative MRI, they are often effectively invisible on intra-operative CT due to minimal contrast between pathological and healthy tissue. This work investigates the feasibility of cross-modality weak supervision for scenarios where pathology is visible in one modality (MRI) but absent in another (CT). We present a hybrid registration-segmentation framework that combines MSCGUNet for inter-modal image registration with a UNet-based segmentation module, enabling registration-assisted pseudo-label generation for CT images. Our evaluation on the CHAOS dataset demonstrates that the pipeline can successfully register and segment healthy liver anatomy, achieving a Dice score of 0.72. However, when applied to clinical data containing tumours, performance degrades substantially (Dice score of 0.16), revealing the fundamental limitations of current registration methods when the target pathology lacks corresponding visual features in the target modality. We analyse the"domain gap"and"feature absence"problems, demonstrating that whilst spatial propagation of labels via registration is feasible for visible structures, segmenting truly invisible pathology remains an open challenge. Our findings highlight that registration-based label transfer cannot compensate for the absence of discriminative features in the target modality, providing important insights for future research in cross-modality medical image analysis. Code an weights are available at: https://github.com/BudhaTronix/Weakly-Supervised-Tumour-Detection

Problem

• cross-modality segmentation
• weakly supervised learning
• invisible tumour
• medical image registration
• feature absence

Innovation

• cross-modality registration
• weakly supervised segmentation
• invisible tumour segmentation
• pseudo-label generation
• feature absence