Low-cost monocular RGB-based 3D structural mapping for horticultural plants via semantic scene completion

Engineering notes

We construct a horticultural 3D semantic scene dataset using an RGB-D sensor, which serves as a benchmark for evaluating our method, while the deployed model remains RGB-only. On our dataset, the method achieves 82.31% occupancy IoU, 84.26% mIoU, and 86.25% precision.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

Precision agriculture increasingly relies on detailed structural information, such as canopy height and canopy volume, to enhance crop health monitoring and operational safety. However, existing methods based on costly LiDAR or RGB-D sensors are often impractical for large-scale deployment in dynamic and unstructured horticultural environments. Furthermore, conventional 2D segmentation and SLAM-based pipelines typically generate sparse, geometrically inconsistent semantic maps which are insufficient for actionable structural analysis in agricultural applications. To overcome these limitations, we propose a monocular 3D structural mapping framework tailored for horticultural plants via semantic scene completion. At inference, the proposed model takes a single RGB image as input and predicts voxel-wise geometry and semantics, from which task-oriented structural maps, including canopy height, canopy volume, and obstacle-aware traversability layers, are derived. Specifically, we first introduce a Depth-Aware Decoder Module that explicitly recovers depth in the spatial domain and fuses 2D-to-3D features, thereby mitigating depth ambiguity and reducing reliance on accurate pose. Second, an NCS-Guided Geometry Encoder is designed to inject normalized depth into voxel positional embeddings, enabling self-attention to perform global relational modeling within a depth-aware geometric coordinate system. In addition, a Global Encoder is utilized to refine local structural details, while an occupancy head produces the final 3D semantic completion outputs. We construct a horticultural 3D semantic scene dataset using an RGB-D sensor, which serves as a benchmark for evaluating our method, while the deployed model remains RGB-only. Extensive quantitative and qualitative experiments are conducted on both the Semantic-KITTI dataset and our dataset. On our dataset, the method achieves 82.31% occupancy IoU, 84.26% mIoU, and 86.25% precision. Beyond voxel-level evaluation, manual field measurements further show canopy height MAE values of 0.019-0.026 m and canopy volume proxy relative errors of 8.4%-11.4%. These results demonstrate the effectiveness of our approach in real-world agricultural scenarios, providing actionable structural insights for crop monitoring and autonomous robotic operations.

Links and sources

• PDF from original source

Comments