MutualVPR: A Mutual Learning Framework for Resolving Supervision Inconsistencies via Adaptive Clustering

2024-12-12

AI Summary

In visual place recognition (VPR), significant appearance variations under viewpoint changes cause inconsistent supervision signals and feature ambiguity—especially under co-directional occlusion—while existing methods rely on manual cropping or directional annotations, limiting generalizability. This paper proposes an unsupervised mutual learning framework: images are grouped by geographic coordinates, and adaptive K-means clustering is jointly optimized with descriptor learning to enable cooperative evolution of viewpoint self-classification and discriminative feature representation. The approach eliminates reliance on directional labels or hand-crafted rules, effectively mitigating dual challenges from viewpoint variance and occlusion. Leveraging the DINOv2 encoder within a dual-branch collaborative training architecture, our method achieves state-of-the-art performance across multiple standard VPR benchmarks, demonstrating substantial improvements in viewpoint generalization and occlusion robustness.

Technology Category

• Computer Vision: Representation Learning for Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Localization, Mapping, and Navigation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Security and Privacy: Data transparency and provenance

Abstract

Visual Place Recognition (VPR) enables robust localization through image retrieval based on learned descriptors. However, drastic appearance variations of images at the same place caused by viewpoint changes can lead to inconsistent supervision signals, thereby degrading descriptor learning. Existing methods either rely on manually defined cropping rules or labeled data for view differentiation, but they suffer from two major limitations: (1) reliance on labels or handcrafted rules restricts generalization capability; (2) even within the same view direction, occlusions can introduce feature ambiguity. To address these issues, we propose MutualVPR, a mutual learning framework that integrates unsupervised view self-classification and descriptor learning. We first group images by geographic coordinates, then iteratively refine the clusters using K-means to dynamically assign place categories without orientation labels. Specifically, we adopt a DINOv2-based encoder to initialize the clustering. During training, the encoder and clustering co-evolve, progressively separating drastic appearance variations of the same place and enabling consistent supervision. Furthermore, we find that capturing fine-grained image differences at a place enhances robustness. Experiments demonstrate that MutualVPR achieves state-of-the-art (SOTA) performance across multiple datasets, validating the effectiveness of our framework in improving view direction generalization, occlusion robustness.

Problem

• Resolving inconsistent supervision signals from drastic appearance variations in Visual Place Recognition
• Eliminating reliance on manual rules or labeled data for view differentiation
• Addressing feature ambiguity caused by occlusions within same view directions

Innovation

• Mutual learning framework integrates unsupervised view classification
• K-means clustering dynamically assigns place categories without labels
• DINOv2 encoder and clustering co-evolve during training