U-Harmony: Enhancing Joint Training for Segmentation Models with Universal Harmonization

2026-01-21

AI Summary

This work addresses the challenge of heterogeneity in multi-institutional medical imaging data—spanning modalities, acquisition protocols, and anatomical targets—which hinders the development of segmentation models that are both generalizable and domain-specific. To this end, the authors propose U-Harmony, a universal harmonization mechanism that integrates a domain-gated head into 3D segmentation networks to sequentially normalize and denormalize feature distributions. This approach effectively mitigates inter-domain discrepancies while preserving intrinsic data characteristics. U-Harmony enables, for the first time, unified joint training across heterogeneous medical datasets and supports seamless extension to new modalities and anatomical classes. Evaluated on multi-institutional brain lesion datasets, the method significantly improves segmentation performance, establishing a new robust and scalable benchmark for 3D medical image segmentation in real-world clinical settings.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

In clinical practice, medical segmentation datasets are often limited and heterogeneous, with variations in modalities, protocols, and anatomical targets across institutions. Existing deep learning models struggle to jointly learn from such diverse data, often sacrificing either generalization or domain-specific knowledge. To overcome these challenges, we propose a joint training method called Universal Harmonization (U-Harmony), which can be integrated into deep learning-based architectures with a domain-gated head, enabling a single segmentation model to learn from heterogeneous datasets simultaneously. By integrating U-Harmony, our approach sequentially normalizes and then denormalizes feature distributions to mitigate domain-specific variations while preserving original dataset-specific knowledge. More appealingly, our framework also supports universal modality adaptation, allowing the seamless learning of new imaging modalities and anatomical classes. Extensive experiments on cross-institutional brain lesion datasets demonstrate the effectiveness of our approach, establishing a new benchmark for robust and adaptable 3D medical image segmentation models in real-world clinical settings.

Problem

• medical image segmentation
• heterogeneous datasets
• domain adaptation
• joint training
• clinical data variability

Innovation

• Universal Harmonization
• domain-gated head
• feature normalization
• modality adaptation
• heterogeneous medical segmentation