Unsupervised Deformable Image Registration with Structural Nonparametric Smoothing

2025-06-12

AI Summary

Unsupervised deformable registration of sparse-textured, large-smooth-region images—such as retinal vasculature—is severely hampered by the aperture problem and large displacements, leading to substantial accuracy degradation. To address this, we propose SmoothProper: a plug-and-play, end-to-end differentiable smoothing constraint module. SmoothProper innovatively integrates a dual-optimization layer with a structured interaction term, enabling simultaneous enforcement of displacement field smoothness and anatomical structure consistency during forward propagation—without requiring manual tuning of regularization hyperparameters. It is model-agnostic and incurs negligible parameter overhead. Evaluated on a high-resolution 2912×2912 retinal vessel dataset, our method achieves a registration error of only 1.88 pixels. To the best of our knowledge, this is the first unsupervised deep deformable registration approach that concurrently mitigates both the aperture problem and large-displacement challenges.

Technology Category

• Computer Vision: Medical and Biological Imaging
• Search and Optimization: Non-convex Optimization
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Learning-based deformable image registration (DIR) accelerates alignment by amortizing traditional optimization via neural networks. Label supervision further enhances accuracy, enabling efficient and precise nonlinear alignment of unseen scans. However, images with sparse features amid large smooth regions, such as retinal vessels, introduce aperture and large-displacement challenges that unsupervised DIR methods struggle to address. This limitation occurs because neural networks predict deformation fields in a single forward pass, leaving fields unconstrained post-training and shifting the regularization burden entirely to network weights. To address these issues, we introduce SmoothProper, a plug-and-play neural module enforcing smoothness and promoting message passing within the network's forward pass. By integrating a duality-based optimization layer with tailored interaction terms, SmoothProper efficiently propagates flow signals across spatial locations, enforces smoothness, and preserves structural consistency. It is model-agnostic, seamlessly integrates into existing registration frameworks with minimal parameter overhead, and eliminates regularizer hyperparameter tuning. Preliminary results on a retinal vessel dataset exhibiting aperture and large-displacement challenges demonstrate our method reduces registration error to 1.88 pixels on 2912x2912 images, marking the first unsupervised DIR approach to effectively address both challenges. The source code will be available at https://github.com/tinymilky/SmoothProper.

Problem

• Addresses aperture challenges in unsupervised image registration
• Solves large-displacement issues in deformable image alignment
• Enhances smoothness and structural consistency in registration fields

Innovation

• Plug-and-play neural module enforces smoothness
• Duality-based optimization layer propagates flow signals
• Model-agnostic integration with minimal parameter overhead