Boosting RGB-D Pear Detection via Depth-Constraint Enhanced Gaussian Prior

Engineering notes

Extensive experiments on pear detection demonstrate that the proposed method achieves an AP50 of 0.961, a precision of 0.941, a recall of 0.951, and an F1-score of 0.942. We further validate generalization on the publicly available KFuji RGB-DS apple dataset, where MMGFormer attains AP50 = 0.927, exceeding the previously reported state-of-the-art (AP50 = 0.901).

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

Accurate pear detection in complex orchard environments is essential for automated harvesting, yet it remains challenging due to frequent occlusion, overlapping fruits, cluttered backgrounds, and highly variable illumination. Although RGB-D sensing provides complementary geometric information beyond RGB imagery, existing methods often fail to fully exploit depth cues and rarely account for the inherently elliptical shape of pears. To address these issues, we propose a multimodal pear detection framework that jointly models RGB and depth information using a Siamese convolutional backbone and a unified Transformer-based fusion architecture. The proposed method contains three key components. First, Gaussian Prior Boxes are introduced to represent pear instances with Gaussian-shaped priors, enabling better alignment with pear contours and more precise localization than conventional rectangular boxes. Second, a Depth-Aware Constraint is designed to enforce depth consistency within the predicted regions, which improves robustness in cluttered orchard scenes. Third, a Robust Cross-Modal Token Exchange strategy is incorporated during training to strengthen feature interaction between RGB and depth modalities and reduce over-reliance on any single modality. Extensive experiments on pear detection demonstrate that the proposed method achieves an AP50 of 0.961, a precision of 0.941, a recall of 0.951, and an F1-score of 0.942. Compared with a strong recent YOLOv8-l RGB baseline (AP50 = 0.918) and a YOLOv8-l RGB-D variant (AP50 = 0.932) trained on the same dataset, our framework yields a notable improvement of +4.3 and +2.9 AP50, respectively. We further validate generalization on the publicly available KFuji RGB-DS apple dataset, where MMGFormer attains AP50 = 0.927, exceeding the previously reported state-of-the-art (AP50 = 0.901). In addition, the model runs at 41.2 FPS, indicating a favorable balance between detection accuracy and real-time performance. These results show the potential of the proposed framework for practical deployment in automated pear harvesting systems.

Links and sources

• PDF from original source

Comments