Deep Shape Regression for Planar Curves with Multimodal Covariates

2026-07-21

AI Summary

This work addresses shape regression for planar open curves under high-dimensional multimodal covariates by proposing a deep shape regression model. The method represents curves as complex-valued functions and employs elastic registration combined with full Procrustes analysis to remove the effects of translation, rotation, scaling, and reparameterization. It introduces modality-specific encoders—such as splines or CNNs—to construct a deep conditional covariance smoother that effectively integrates heterogeneous covariates, including scalars and images. By innovatively combining deep learning with shape statistics, the approach inherently respects shape invariance and accommodates sparsely and irregularly sampled curves. Experiments demonstrate that the model accurately recovers conditional mean shapes in simulations and successfully replicates established covariate effects on hippocampal contours in the ADNI dataset, confirming its validity and practical utility.

Abstract

The shape of a planar curve is the geometric information that remains once translation, rotation, scale and reparametrisation are removed and is of interest in many health applications, e.g. in neuroimaging. We propose a deep shape regression model for open planar curves that admits multimodal and high-dimensional covariates. Representing curves as complex-valued functions, we show that the conditional full Procrustes mean is the leading eigenfunction of the conditional covariance. To estimate this covariance surface, we propose a novel deep conditional covariance smoother with modality-specific encoders - e.g. splines for scalar covariates and convolutional networks for images, which classical spline smoothers cannot accommodate. Our model is by construction invariant to the translation, rotation and scaling of the input curves and handles sparsely and irregularly sampled curves. We further provide an algorithm for elastic mean estimation that also removes parametrisation by iterating covariance smoothing, rotational alignment and parametrisation alignment. We illustrate the method on simulated outlines with known conditional mean and multimodal covariates, and give a first application to hippocampal outlines from the ADNI cohort, recovering covariate effects consistent with the literature. Code is available at https://github.com/mpff/dnn-shapes.

Problem

• shape regression
• planar curves
• multimodal covariates
• Procrustes mean
• conditional covariance

Innovation

• deep shape regression
• multimodal covariates
• conditional covariance smoother
• elastic mean estimation
• Procrustes mean