BEV-Based Multimodal Fusion and Multi-Vehicle Collaborative Perception in Autonomous Driving

Engineering notes

Key topics: autonomous driving, bev, lidar, sensor fusion, multi-sensor fusion, perception. See the paper for implementation details and experimental results.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

Single-vehicle perception systems have achieved impressive results by fusing cameras and LiDAR to exploit complementary semantic and geometric cues. However, these systems remain fundamentally limited when confronting visual occlusion: objects obscured by large vehicles, infrastructure, or dynamic obstacles may evade detection entirely. To address this, we propose MPRF, a Bird's-Eye View (BEV)-based multi-vehicle collaborative perception framework. Specifically, we first achieve accurate transformations from both camera and LiDAR modalities into the BEV representation, ensuring precise spatial alignment; next, we employ flexible multi-sensor fusion strategies-including feature concatenation and confidence-weighted fusion-within the BEV domain to seamlessly combine semantic and geometric cues; finally, we introduce a novel multi-vehicle perception fusion algorithm, which aggregates overlapping detections across vehicles through confidence-weighted bounding-box averaging enhanced with a consensus-aware rescaling mechanism. Evaluated on simulated and real-world scenarios, our method demonstrates significant performance gains-over $\mathbf{1 5 \%}$ higher $\mathbf{m A P}$ and $\mathbf{8 c m}$ lower localization error-compared to per-vehicle baselines and common ensemble methods.

Links and sources

• Official / arXiv page

Comments