TARDis: Time Attenuated Representation Disentanglement for Incomplete Multi-Modal Tumor Segmentation and Classification

2025-12-04

AI Summary

In contrast-enhanced CT, radiation dose constraints often lead to missing multiphase acquisitions (“missing modalities”), yet existing methods neglect the temporal continuity of hemodynamic processes. Method: We propose a physics-informed temporal attenuation modeling paradigm that, for the first time, formalizes missing phases as missing samples along a continuous perfusion curve, explicitly decoupling time-invariant anatomical structure from time-varying enhancement dynamics. Our dual-path architecture employs a quantized dictionary to extract stable anatomical representations and a conditional variational autoencoder to model perfusion temporal evolution, enabling physiologically plausible reconstruction of missing phases. Contribution/Results: Evaluated on multiple private and public abdominal CT datasets, our method significantly outperforms state-of-the-art approaches. Even under extreme sparsity (e.g., only 1–2 acquired phases), it maintains high-accuracy tumor segmentation and classification—effectively supporting low-dose clinical imaging requirements.

Technology Category

• Computer Vision: Multi-modal Vision
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Tumor segmentation and diagnosis in contrast-enhanced Computed Tomography (CT) rely heavily on the physiological dynamics of contrast agents. However, obtaining a complete multi-phase series is often clinically unfeasible due to radiation concerns or scanning limitations, leading to the "missing modality" problem. Existing deep learning approaches typically treat missing phases as absent independent channels, ignoring the inherent temporal continuity of hemodynamics. In this work, we propose Time Attenuated Representation Disentanglement (TARDis), a novel physics-aware framework that redefines missing modalities as missing sample points on a continuous Time-Attenuation Curve. TARDis explicitly disentangles the latent feature space into a time-invariant static component (anatomy) and a time-dependent dynamic component (perfusion). We achieve this via a dual-path architecture: a quantization-based path using a learnable embedding dictionary to extract consistent anatomical structures, and a probabilistic path using a Conditional Variational Autoencoder to model dynamic enhancement conditioned on the estimated scan time. This design allows the network to hallucinate missing hemodynamic features by sampling from the learned latent distribution. Extensive experiments on a large-scale private abdominal CT dataset (2,282 cases) and two public datasets demonstrate that TARDis significantly outperforms state-of-the-art incomplete modality frameworks. Notably, our method maintains robust diagnostic performance even in extreme data-sparsity scenarios, highlighting its potential for reducing radiation exposure while maintaining diagnostic precision.

Problem

• Addresses incomplete multi-phase CT data for tumor segmentation and classification.
• Models missing modalities as points on continuous time-attenuation curves.
• Disentangles static anatomical and dynamic perfusion features to hallucinate missing data.

Innovation

• Redefines missing modalities as missing points on continuous time-attenuation curve
• Disentangles features into time-invariant anatomy and time-dependent perfusion components
• Uses dual-path architecture with quantization and conditional variational autoencoder