Cross-modal Fundus Image Registration under Large FoV Disparity

2025-12-14

AI Summary

Cross-modal registration between optical coherence tomography angiography (OCTA) and wide-field color fundus photography (wfCFP) fails due to severe field-of-view (FOV) mismatch—OCTA captures a small FOV while wfCFP covers a large FOV. Method: This paper proposes CARe, a learning-free registration framework. First, an anatomy-guided intelligent cropping strategy coarsely aligns the large-FOV wfCFP to the small-FOV OCTA region using retinal structural priors. Second, a dual-fitting alignment module integrates RANSAC-based robust estimation with polynomial coordinate transformation to enhance both accuracy and robustness of spatial mapping. Contribution/Results: CARe is lightweight, interpretable, and seamlessly embeddable into conventional registration pipelines. Evaluated on a self-collected dataset of 60 OCTA–wfCFP image pairs, CARe outperforms state-of-the-art small-FOV registration methods, reducing mean registration error by 32.7%. It establishes a reliable, geometrically consistent foundation for multimodal retinal analysis.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Security and Privacy: Data transparency and provenance
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

Previous work on cross-modal fundus image registration (CMFIR) assumes small cross-modal Field-of-View (FoV) disparity. By contrast, this paper is targeted at a more challenging scenario with large FoV disparity, to which directly applying current methods fails. We propose Crop and Alignment for cross-modal fundus image Registration(CARe), a very simple yet effective method. Specifically, given an OCTA with smaller FoV as a source image and a wide-field color fundus photograph (wfCFP) as a target image, our Crop operation exploits the physiological structure of the retina to crop from the target image a sub-image with its FoV roughly aligned with that of the source. This operation allows us to re-purpose the previous small-FoV-disparity oriented methods for subsequent image registration. Moreover, we improve spatial transformation by a double-fitting based Alignment module that utilizes the classical RANSAC algorithm and polynomial-based coordinate fitting in a sequential manner. Extensive experiments on a newly developed test set of 60 OCTA-wfCFP pairs verify the viability of CARe for CMFIR.

Problem

• Addresses cross-modal fundus image registration with large field-of-view disparity
• Proposes a crop and alignment method to enable registration of OCTA and wide-field fundus images
• Improves spatial transformation using RANSAC and polynomial fitting for accurate alignment

Innovation

• Crop operation aligns FoV using retinal structure
• Double-fitting alignment with RANSAC and polynomial fitting
• Enables reuse of small-FoV methods for large disparity