Multi-Level Bidirectional Decoder Interaction for Uncertainty-Aware Breast Ultrasound Analysis

2026-03-01

AI Summary

This work addresses the challenge in breast ultrasound analysis where conventional multi-task learning suffers from task interference and inflexible collaboration strategies that fail to account for varying prediction difficulties across samples. To overcome this, the authors propose a novel multi-task framework that enables dynamic, adaptive cooperation between segmentation and classification tasks during spatial reconstruction. The approach leverages a bidirectional interaction mechanism across hierarchical decoders and an uncertainty-aware attention module guided by feature activation variance. This design eliminates the need for manual hyperparameter tuning and facilitates layer-wise and sample-wise task balancing while integrating multi-scale contextual information. Extensive experiments on multiple public datasets demonstrate its effectiveness, achieving a lesion segmentation IoU of 74.5% and a classification accuracy of 90.6% on the BUSI dataset. Ablation studies further validate the contribution of the proposed interaction mechanism.

Technology Category

• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Breast ultrasound interpretation requires simultaneous lesion segmentation and tissue classification. However, conventional multi-task learning approaches suffer from task interference and rigid coordination strategies that fail to adapt to instance-specific prediction difficulty. We propose a multi-task framework addressing these limitations through multi-level decoder interaction and uncertainty-aware adaptive coordination. Task Interaction Modules operate at all decoder levels, establishing bidirectional segmentation-classification communication during spatial reconstruction through attention weighted pooling and multiplicative modulation. Unlike prior single-level or encoder-only approaches, this multi-level design captures scale specific task synergies across semantic-to-spatial scales, producing complementary task interaction streams. Uncertainty-Proxy Attention adaptively weights base versus enhanced features at each level using feature activation variance, enabling per-level and per-sample task balancing without heuristic tuning. To support instance-adaptive prediction, multi-scale context fusion captures morphological cues across varying lesion sizes. Evaluation on multiple publicly available breast ultrasound datasets demonstrates competitive performance, including 74.5% lesion IoU and 90.6% classification accuracy on BUSI dataset. Ablation studies confirm that multi-level task interaction provides significant performance gains, validating that decoder-level bidirectional communication is more effective than conventional encoder-only parameter sharing. The code is available at: https://github.com/C-loud-Nine/Uncertainty-Aware-Multi-Level-Decoder-Interaction.

Problem

• breast ultrasound
• multi-task learning
• task interference
• uncertainty-aware
• lesion segmentation

Innovation

• multi-level decoder interaction
• uncertainty-aware learning
• bidirectional task communication
• adaptive feature weighting
• multi-task ultrasound analysis