An Improved Multi-Modal Fusion 3D Object Detection Algorithm Based on Point Density

Engineering notes

Experimental results on the KITTI dataset show that our proposed 3D object detection method outperforms existing approaches.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

LiDAR has emerged as a widely used sensor for 3D object detection in autonomous driving. However, the point cloud data generated by LiDAR is typically sparse and irregular, necessitating the adoption of cross-modal detection approaches to enhance detection accuracy and stability. The distinct representations of point clouds and images pose challenges for effective data fusion, often resulting in suboptimal performance. In this study, we introduce a novel multi-modal, density-aware 3D object detection framework, PVNet, which leverages virtual point clouds generated through depth completion to overcome fusion difficulties. To mitigate errors introduced by inaccuracies in depth completion, we propose a Virtual Point Cloud Enhancement Block. This block first extracts features from both the original point cloud and virtual points using sparse convolution, and then combines these features with our centroid shift weighting module, reducing interference from virtual point noise. Additionally, a self-attention mechanism is employed for more effective feature fusion. To fully exploit the complementary information from both LiDAR and camera data, and improve the detection of distant and small objects, we incorporate a multi-scale attention fusion mechanism. Experimental results on the KITTI dataset show that our proposed 3D object detection method outperforms existing approaches.

Links and sources

• Official / arXiv page
• PDF from original source

Comments