SSHNet: Unsupervised Cross-modal Homography Estimation via Problem Redefinition and Split Optimization

2024-09-26

AI Summary

This paper addresses the challenging problem of unsupervised cross-modal (e.g., optical–SAR) image homography estimation. To tackle this, we propose a novel paradigm that redefines and decouples the task into two supervised subtasks: homography estimation and modality translation. We introduce a source-consistent feature space supervision mechanism to enhance cross-modal feature alignment and incorporate knowledge distillation to improve model generalization and parameter efficiency. Our framework integrates state-of-the-art architectures—including IHN, MHN, and LocalTrans—into a unified design. Evaluated on the OPT-SAR benchmark, our SSHNet-IHN achieves substantial gains over existing unsupervised methods. Compared to supervised counterparts MHN and LocalTrans, it reduces the mean angular correspondence error (MACE) by 47.4% and 85.8%, respectively, demonstrating superior accuracy, robustness, and computational efficiency.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

We propose a novel unsupervised cross-modal homography estimation learning framework, named Split Supervised Homography estimation Network (SSHNet). SSHNet redefines the unsupervised cross-modal homography estimation into two supervised sub-problems, each addressed by its specialized network: a homography estimation network and a modality transfer network. To realize stable training, we introduce an effective split optimization strategy to train each network separately within its respective sub-problem. We also formulate an extra homography feature space supervision to enhance feature consistency, further boosting the estimation accuracy. Moreover, we employ a simple yet effective distillation training technique to reduce model parameters and improve cross-domain generalization ability while maintaining comparable performance. The training stability of SSHNet enables its cooperation with various homography estimation architectures. Experiments reveal that the SSHNet using IHN as homography estimation network, namely SSHNet-IHN, outperforms previous unsupervised approaches by a significant margin. Even compared to supervised approaches MHN and LocalTrans, SSHNet-IHN achieves 47.4% and 85.8% mean average corner errors (MACEs) reduction on the challenging OPT-SAR dataset.

Problem

• Unsupervised cross-modal homography estimation
• Split optimization strategy
• Enhanced feature consistency and accuracy

Innovation

• Unsupervised cross-modal homography estimation
• Split optimization strategy training
• Distillation technique reduces parameters