Geodesic Prototype Matching via Diffusion Maps for Interpretable Fine-Grained Recognition

2025-09-21

AI Summary

In fine-grained recognition, Euclidean distance fails to capture the nonlinear manifold structure of deep features, leading to inaccurate semantic discrepancy modeling. To address this, we propose a manifold-aware prototype matching framework. Our core innovation is the first integration of diffusion maps with differentiable Nyström interpolation to construct a dynamically aligned manifold prototype space; compact, periodically updated landmark sets enable efficient geometric alignment and scalable prototype learning. The method is seamlessly embedded into deep feature extractors and supports end-to-end differentiable optimization. On CUB-200-2011 and Stanford Cars, it significantly outperforms Euclidean prototype baselines. Learned prototypes precisely localize semantically consistent regions, achieving both improved classification accuracy and strong interpretability.

Technology Category

• Machine Learning: Learning with Manifolds
• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

Nonlinear manifolds are widespread in deep visual features, where Euclidean distances often fail to capture true similarity. This limitation becomes particularly severe in prototype-based interpretable fine-grained recognition, where subtle semantic distinctions are essential. To address this challenge, we propose a novel paradigm for prototype-based recognition that anchors similarity within the intrinsic geometry of deep features. Specifically, we distill the latent manifold structure of each class into a diffusion space and introduce a differentiable Nyström interpolation, making the geometry accessible to both unseen samples and learnable prototypes. To ensure efficiency, we employ compact per-class landmark sets with periodic updates. This design keeps the embedding aligned with the evolving backbone, enabling fast and scalable inference. Extensive experiments on the CUB-200-2011 and Stanford Cars datasets show that our GeoProto framework produces prototypes focusing on semantically aligned parts, significantly outperforming Euclidean prototype networks.

Problem

• Measuring feature similarity on nonlinear manifolds using Euclidean distances
• Capturing subtle semantic distinctions in fine-grained recognition tasks
• Aligning prototype learning with intrinsic geometry of deep features

Innovation

• Uses diffusion maps to model feature manifolds
• Introduces differentiable Nyström interpolation for geometry
• Employs compact landmarks with periodic updates