UAV-DETR: Efficient End-to-End Object Detection for Unmanned Aerial Vehicle Imagery

2025-01-03

AI Summary

To address the challenges of labor-intensive hyperparameter tuning, poor generalizability of generic detectors, and insufficient robustness to small objects and dense scenes in UAV-based object detection (UAV-OD), this paper proposes an end-to-end learnable DETR framework. Our method introduces three novel components: (1) multi-scale frequency-domain enhanced feature fusion, (2) frequency-aware downsampling, and (3) semantic alignment calibration—enabling joint spatial-frequency modeling and cross-path feature co-optimization. It integrates feature pyramids, discrete cosine transform (DCT)-based frequency representations, learnable downsampling, and cross-attention-driven semantic alignment. On VisDrone, our approach achieves +3.1% AP and +4.2% AP₅₀ over strong baselines; on UAVVaste, it consistently outperforms existing methods, demonstrating superior generalization and practical deployability for real-world UAV applications.

Technology Category

• Computer Vision: Object Detection & Categorization
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

Unmanned aerial vehicle object detection (UAV-OD) has been widely used in various scenarios. However, most existing UAV-OD algorithms rely on manually designed components, which require extensive tuning. End-to-end models that do not depend on such manually designed components are mainly designed for natural images, which are less effective for UAV imagery. To address such challenges, this paper proposes an efficient detection transformer (DETR) framework tailored for UAV imagery, i.e., UAV-DETR. The framework includes a multi-scale feature fusion with frequency enhancement module, which captures both spatial and frequency information at different scales. In addition, a frequency-focused down-sampling module is presented to retain critical spatial details during down-sampling. A semantic alignment and calibration module is developed to align and fuse features from different fusion paths. Experimental results demonstrate the effectiveness and generalization of our approach across various UAV imagery datasets. On the VisDrone dataset, our method improves AP by 3.1% and $ ext{AP}_{50}$ by 4.2% over the baseline. Similar enhancements are observed on the UAVVaste dataset. The project page: https://github.com/ValiantDiligent/UAV-DETR

Problem

• UAV-OD
• Adaptability
• Precision

Innovation

• UAV-DETR
• Multi-Angle Image Detection
• Drone Photography Optimization