WeedVision: Multi-Stage Growth and Classification of Weeds using DETR and RetinaNet for Precision Agriculture

2025-02-16

AI Summary

To address the challenge of fine-grained weed identification in precision agriculture, this paper proposes the first dual-task framework jointly modeling growth stages and species recognition. It enables end-to-end detection and classification of 16 economically significant weed species across an 11-week growth cycle—comprising 174 distinct stage-species classes. We introduce the largest publicly available temporally annotated weed dataset to date (203,000 images) and empirically validate, for the first time, the positive correlation between growth-stage awareness and detection accuracy. Our approach employs a RetinaNet detector with a ResNeXt101 backbone, achieving 0.904 mAP on the test set—outperforming DETR by +3.2%—while maintaining high recall and real-time inference speed (7.28 FPS). Notably, model performance improves steadily with plant maturity, demonstrating robustness across developmental stages. This work establishes a practical, deployable technical foundation for intelligent, stage-aware weed management systems.

Technology Category

• Computer Vision: Object Detection & Categorization
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Planning, Routing, and Scheduling: Activity and Plan Recognition

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Web search models and ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Weed management remains a critical challenge in agriculture, where weeds compete with crops for essential resources, leading to significant yield losses. Accurate detection of weeds at various growth stages is crucial for effective management yet challenging for farmers, as it requires identifying different species at multiple growth phases. This research addresses these challenges by utilizing advanced object detection models, specifically, the Detection Transformer (DETR) with a ResNet50 backbone and RetinaNet with a ResNeXt101 backbone, to identify and classify 16 weed species of economic concern across 174 classes, spanning their 11 weeks growth stages from seedling to maturity. A robust dataset comprising 203,567 images was developed, meticulously labeled by species and growth stage. The models were rigorously trained and evaluated, with RetinaNet demonstrating superior performance, achieving a mean Average Precision (mAP) of 0.907 on the training set and 0.904 on the test set, compared to DETR's mAP of 0.854 and 0.840, respectively. RetinaNet also outperformed DETR in recall and inference speed of 7.28 FPS, making it more suitable for real time applications. Both models showed improved accuracy as plants matured. This research provides crucial insights for developing precise, sustainable, and automated weed management strategies, paving the way for real time species specific detection systems and advancing AI-assisted agriculture through continued innovation in model development and early detection accuracy.

Problem

• Accurate weed detection across growth stages
• Classification of 16 weed species
• Real-time AI-assisted weed management

Innovation

• Utilizes DETR with ResNet50 backbone
• Employs RetinaNet with ResNeXt101 backbone
• Achieves high precision in weed classification