LP-DETR: Layer-wise Progressive Relations for Object Detection

2025-02-07

AI Summary

To address DETR’s slow convergence and oversimplified spatial relationship modeling, this work proposes an inter-layer progressive multi-scale relational modeling mechanism: adaptive fusion of local, medium-range, and global spatial dependencies across decoder layers, coupled with a relation-aware progressive self-attention module that reveals the attention evolution pattern—shallow layers favor local cues while deeper layers emphasize global context. By optimizing decoder architecture and embedding multi-scale relational priors, the method significantly enhances spatial dependency modeling capacity. Experiments demonstrate rapid convergence and superior accuracy: 52.3% AP on COCO after only 12 epochs (ResNet-50 backbone), 52.5% AP after 24 epochs, and 58.0% AP when substituting Swin-Large. The core contribution lies in the first integration of relation awareness and inter-layer progression into the DETR decoder, enabling dynamic, hierarchical spatial dependency modeling.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Reasoning under Uncertainty: Relational Probabilistic Models
• Machine Learning: Deep Neural Architectures and Foundation Models

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

This paper presents LP-DETR (Layer-wise Progressive DETR), a novel approach that enhances DETR-based object detection through multi-scale relation modeling. Our method introduces learnable spatial relationships between object queries through a relation-aware self-attention mechanism, which adaptively learns to balance different scales of relations (local, medium and global) across decoder layers. This progressive design enables the model to effectively capture evolving spatial dependencies throughout the detection pipeline. Extensive experiments on COCO 2017 dataset demonstrate that our method improves both convergence speed and detection accuracy compared to standard self-attention module. The proposed method achieves competitive results, reaching 52.3% AP with 12 epochs and 52.5% AP with 24 epochs using ResNet-50 backbone, and further improving to 58.0% AP with Swin-L backbone. Furthermore, our analysis reveals an interesting pattern: the model naturally learns to prioritize local spatial relations in early decoder layers while gradually shifting attention to broader contexts in deeper layers, providing valuable insights for future research in object detection.

Problem

• Enhances DETR-based object detection
• Models multi-scale spatial relationships
• Improves convergence and detection accuracy

Innovation

• Layer-wise Progressive DETR
• Relation-aware self-attention mechanism
• Multi-scale relation modeling