Technology Category
Computer Vision: SegmentationIntelligent Robots: Multimodal Perception & Sensor FusionMachine Learning: Calibration & Uncertainty Quantification
Deep Learning-Based Direct Leaf Area Estimation using Two RGBD Datasets for Model Development
🤖 AI Summary
This study addresses the challenge of accurate single-leaf area estimation for crop phenotyping directly from RGB-D images. We propose the first end-to-end deep learning method that jointly optimizes leaf segmentation and area regression via a dual-backbone architecture, specifically adapting Mask R-CNN to natively process RGB-D inputs—bypassing conventional post-segmentation processing. To enhance segmentation fidelity under agricultural constraints, we introduce an Intersection-over-Area (IoA)-aware segmentation evaluation metric and an agriculture-guided, agile hyperparameter tuning strategy. Five-fold cross-validation demonstrates strong performance: on unseen detached-leaf data, segmentation achieves F1 = 1.0 (IoA ≥ 0.9) and area estimation attains R² = 0.81; on unseen whole-plant images, R² = 0.57. These results validate both the method’s accuracy and its robust cross-scenario generalization capability—from detached leaves to intact plants—establishing a new benchmark for direct, sensor-native leaf phenotyping.
Technology Category
Application Category
📝 Abstract
Estimation of a single leaf area can be a measure of crop growth and a phenotypic trait to breed new varieties. It has also been used to measure leaf area index and total leaf area. Some studies have used hand-held cameras, image processing 3D reconstruction and unsupervised learning-based methods to estimate the leaf area in plant images. Deep learning works well for object detection and segmentation tasks; however, direct area estimation of objects has not been explored. This work investigates deep learning-based leaf area estimation, for RGBD images taken using a mobile camera setup in real-world scenarios. A dataset for attached leaves captured with a top angle view and a dataset for detached single leaves were collected for model development and testing. First, image processing-based area estimation was tested on manually segmented leaves. Then a Mask R-CNN-based model was investigated, and modified to accept RGBD images and to estimate the leaf area. The detached-leaf data set was then mixed with the attached-leaf plant data set to estimate the single leaf area for plant images, and another network design with two backbones was proposed: one for segmentation and the other for area estimation. Instead of trying all possibilities or random values, an agile approach was used in hyperparameter tuning. The final model was cross-validated with 5-folds and tested with two unseen datasets: detached and attached leaves. The F1 score with 90% IoA for segmentation result on unseen detached-leaf data was 1.0, while R-squared of area estimation was 0.81. For unseen plant data segmentation, the F1 score with 90% IoA was 0.59, while the R-squared score was 0.57. The research suggests using attached leaves with ground truth area to improve the results.
Problem
Research questions and friction points this paper is trying to address.
Develop deep learning model for leaf area estimation.
Use RGBD images for accurate leaf area measurement.
Improve segmentation and area estimation in plant images.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Mask R-CNN model adapted for RGBD images
Two-backbone network for segmentation and area estimation
Agile hyperparameter tuning for model optimization
🔎 Similar Papers
No similar papers found.
