NISF++: Geometrically-grounded implicit representations of 3D+time cardiac function from 2D short- and long-axis MR views

2026-08-01

AI Summary

Current deep learning approaches struggle to construct spatiotemporally coherent 3D+time cardiac representations from multiplanar 2D cardiac MRI and lack robust motion correction capabilities. This work proposes a unified 4D cardiac modeling framework based on neural implicit segmentation functions, which jointly learns short-axis and long-axis slices from arbitrary orientations and incorporates learnable rigid transformation parameters to correct for respiratory and patient motion. The method enables continuous interpolation of 4D intensity and segmentation at arbitrary spatial and temporal resolutions while preserving geometric consistency. Evaluated on 120 cases from the UK Biobank, the approach achieves segmentation performance comparable to state-of-the-art methods—where failures primarily stem from annotation limitations—and demonstrates significantly improved anatomical plausibility along with quantitative and qualitative enhancements in slice alignment.

Abstract

Clinical acquisition in cardiac magnetic resonance (CMR) imaging involves obtaining cross-sectional planes of the heart along the radial and longitudinal directions. Despite these planes being 2D cross-sectional images of the heart, radiologists understand the 3D spatial and continuous temporal nature of the organ being imaged. The same can not be said about the conventional deep learning architectures used to process CMR images, which rely on in-plane and grid-based operations, and are hence unable to organically integrate information from all imaging planes. This paper builds upon previous work on neural implicit segmentation functions (NISF) to overcome unaddressed challenges in cardiac function modeling in the CMR domain. For a given subject, our architecture builds a shared 3D+time representations from all available acquisition planes regardless of orientation. By design, predictions along any imaging plane orientation are cross-sections of the same 3D representation, leading to spatio-temporal consistency across all slices. Moreover, our architecture makes the rotation and translation parameters of imaging planes learnable, allowing us to correct for the commonplace respiratory and patient motion between slice acquisitions under a rigid assumption. Furthermore, interpolation of intensities and segmentation can be performed in 4D at any desired resolution. We perform our study on a 120 subject sub-cohort of CMR imaging data from the UK-Biobank. Our in-plane segmentation performance is on-par with existing CMR segmentation methods and explore how the majority of failure cases arise from limitations in the ground-truth segmentation, for which our representations make predictions with better anatomical accuracy than its original training data. We also evaluate our motion-correction capabilities, displaying quantitative and qualitative improvements in slice alignment.

Problem

• cardiac MRI
• 3D+time representation
• multi-plane integration
• motion correction
• spatio-temporal consistency

Innovation

• neural implicit representation
• 3D+time cardiac modeling
• motion correction
• multi-planar CMR fusion
• spatio-temporal consistency