Fusing Bird’s Eye View LIDAR Point Cloud and Front View Camera Image for 3D Object Detection

Engineering notes

Key topics: autonomous driving, 3d object detection, object detection, lidar, point cloud, camera-lidar fusion, kitti. See the paper for implementation details and experimental results.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

We propose a new method for fusing LIDAR point cloud and camera-captured images in deep convolutional neural networks (CNN). The proposed method constructs a new layer called sparse non-homogeneous pooling layer to transform features between bird’s eye view and front view. The sparse point cloud is used to construct the mapping between the two views. The pooling layer allows efficient fusion of the multi-view features at any stage of the network. This is favorable for 3D object detection using camera-LIDAR fusion for autonomous driving. A corresponding one-stage detector is designed and tested on the KITTI bird’s eye view object detection dataset, which produces 3D bounding boxes from the bird’s eye view map. The fusion method shows significant improvement on both speed and accuracy of the pedestrian detection over other fusion-based object detection networks.

Links and sources

• Official / arXiv page

Comments