Self-supervised structured object representation learning

2025-08-27

AI Summary

Self-supervised learning (SSL) excels at global image understanding but struggles to model multi-scale, structured object representations in complex scenes. To address this, we propose ProtoScale—a novel module that enables fine-grained instance separation and hierarchical structural modeling without relying on global cropping. ProtoScale integrates semantic-clustering-guided multi-scale feature aggregation with context-preserving data augmentation. It jointly optimizes semantic grouping, instance discrimination, and cross-scale contextual modeling, thereby significantly enhancing structured representation learning. Evaluated on a joint COCO and UA-DETRAC benchmark, our approach achieves state-of-the-art downstream object detection performance using only minimal annotations and few fine-tuning epochs. This demonstrates the effectiveness and generalizability of structured SSL representations for dense prediction tasks.

Technology Category

• Computer Vision: Representation Learning for Vision
• Machine Learning: Structured Learning
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Self-supervised learning (SSL) has emerged as a powerful technique for learning visual representations. While recent SSL approaches achieve strong results in global image understanding, they are limited in capturing the structured representation in scenes. In this work, we propose a self-supervised approach that progressively builds structured visual representations by combining semantic grouping, instance level separation, and hierarchical structuring. Our approach, based on a novel ProtoScale module, captures visual elements across multiple spatial scales. Unlike common strategies like DINO that rely on random cropping and global embeddings, we preserve full scene context across augmented views to improve performance in dense prediction tasks. We validate our method on downstream object detection tasks using a combined subset of multiple datasets (COCO and UA-DETRAC). Experimental results show that our method learns object centric representations that enhance supervised object detection and outperform the state-of-the-art methods, even when trained with limited annotated data and fewer fine-tuning epochs.

Problem

• Self-supervised learning captures structured object representations
• Preserves full scene context across augmented views
• Enhances object detection with limited annotated data

Innovation

• Semantic grouping and instance separation
• ProtoScale module for multi-scale elements
• Full scene context preservation across views