LMPNet for Weakly-supervised Keypoint Discovery

2025-07-03

AI Summary

This paper addresses weakly supervised semantic keypoint discovery using only image-level class labels as supervision. The proposed discriminative learning framework eliminates the need for explicit keypoint annotations by explicitly repurposing intermediate CNN filters as keypoint detectors. A funnel-shaped max-pooling (LMP) layer is introduced to enhance filter responsiveness to non-repetitive local patterns, while a learnable clustering layer enables automatic grouping of detected keypoints. To improve interpretability, an attention-based masking mechanism and visualization-guided filter selection strategy are incorporated. Evaluated on multiple benchmarks, the method achieves accuracy comparable to fully supervised pose estimation models. It demonstrates strong robustness to object pose variations and reliably localizes consistent semantic keypoints across diverse poses. Overall, the approach significantly advances the accuracy and structural coherence of weakly supervised keypoint discovery.

Technology Category

• Computer Vision: Representation Learning for Vision
• Machine Learning: Semi-Supervised Learning
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In this work, we explore the task of semantic object keypoint discovery weakly-supervised by only category labels. This is achieved by transforming discriminatively-trained intermediate layer filters into keypoint detectors. We begin by identifying three preferred characteristics of keypoint detectors: (i) spatially sparse activations, (ii) consistency and (iii) diversity. Instead of relying on hand-crafted loss terms, a novel computationally-efficient leaky max pooling (LMP) layer is proposed to explicitly encourage final conv-layer filters to learn "non-repeatable local patterns" that are well aligned with object keypoints. Informed by visualizations, a simple yet effective selection strategy is proposed to ensure consistent filter activations and attention mask-out is then applied to force the network to distribute its attention to the whole object instead of just the most discriminative region. For the final keypoint prediction, a learnable clustering layer is proposed to group keypoint proposals into keypoint predictions. The final model, named LMPNet, is highly interpretable in that it directly manipulates network filters to detect predefined concepts. Our experiments show that LMPNet can (i) automatically discover semantic keypoints that are robust to object pose and (ii) achieves strong prediction accuracy comparable to a supervised pose estimation model.

Problem

• Discover semantic keypoints using only category labels
• Transform layer filters into keypoint detectors efficiently
• Ensure keypoint diversity and consistency without manual losses

Innovation

• Leaky max pooling for non-repeatable patterns
• Selection strategy for consistent filter activations
• Learnable clustering for keypoint grouping