Pruning Sparse Tensor Neural Networks Enables Deep Learning for 3D Ultrasound Localization Microscopy

2024-02-14

IEEE Transactions on Image Processing

AI Summary

To address the memory explosion hindering deep learning-based 3D ultrasound localization microscopy (ULM) and its degraded trajectory detection performance under high microbubble concentrations, this work introduces sparse tensor neural networks into 3D ULM reconstruction for the first time. We propose an end-to-end architecture based on coordinate-aware sparse convolutions (e.g., MinkowskiNet-style), leveraging the intrinsic sparsity of microbubble trajectories via sparse tensor representations and sparse tensor convolutions—preserving representational capacity while drastically reducing memory consumption. Experiments demonstrate a two-order-of-magnitude memory reduction in 3D scenarios and significantly improved localization accuracy and robustness over conventional ULM at high concentrations. In 2D validation, memory usage is halved with only marginal accuracy degradation. This approach breaks the current deep learning ULM bottleneck confined to 2D, enabling higher microbubble concentrations and shorter acquisition times.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Computer Vision: 3D Computer Vision
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Ultrasound Localization Microscopy (ULM) is a non-invasive technique that allows for the imaging of micro-vessels in vivo, at depth and with a resolution on the order of ten microns. ULM is based on the sub-resolution localization of individual microbubbles injected in the bloodstream. Mapping the whole angioarchitecture requires the accumulation of microbubbles trajectories from thousands of frames, typically acquired over a few minutes. ULM acquisition times can be reduced by increasing the microbubble concentration, but requires more advanced algorithms to detect them individually. Several deep learning approaches have been proposed for this task, but they remain limited to 2D imaging, in part due to the associated large memory requirements. Herein, we propose the use of sparse tensor neural networks to enable deep learning-based 3D ULM by improving memory scalability with increased dimensionality. We study several approaches to efficiently convert ultrasound data into a sparse format and study the impact of the associated loss of information. When applied in 2D, the sparse formulation reduces the memory requirements by a factor 2 at the cost of a small reduction of performance when compared against dense networks. In 3D, the proposed approach reduces memory requirements by two order of magnitude while largely outperforming conventional ULM in high concentration settings. We show that Sparse Tensor Neural Networks in 3D ULM allow for the same benefits as dense deep learning based method in 2D ULM i.e. the use of higher concentration in silico and reduced acquisition time.

Problem

• Reducing memory usage for 3D ultrasound localization microscopy
• Enabling deep learning in 3D imaging with sparse tensor networks
• Improving microbubble detection efficiency to shorten acquisition times

Innovation

• Sparse tensor networks reduce 3D memory usage
• Efficient ultrasound data conversion to sparse format
• Enables 3D deep learning with high concentration microbubbles