FDARC: Frequency-Aware and Depth Association Radar–Camera Fusion

Engineering notes

Extensive experiments conducted on the nuScenes dataset show that FDARC achieves state-of-the-art performance in 3D detection tasks, markedly outperforming baseline models on the nuScenes val set using a ResNet-50 backbone, which attains 53.5% nuScenes Detection Score (NDS) and 44.7% mean Average Precision (mAP).

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

Autonomous driving necessitates a robust 3D perception system that includes accurate object detection, tracking, and segmentation. While recent low-cost camera-based methods have demonstrated promising results, these systems are prone to performance degradation under poor lighting conditions or adverse weather, resulting in considerable localization errors. In this paper, we present a novel approach called Frequency-aware Depth Association Radar-Camera (FDARC) Fusion. This method aims to generate semantically rich and spatially accurate Bird’s-Eye-View (BEV) feature maps by integrating data from both camera and radar sensors. Initially, the image features are enhanced using frequency-aware techniques. Subsequently, these features are transformed into BEV representation with the assistance of depth information estimated from both sensor modalities and radar measurements. This process, known as Depth Association (DA), facilitates more precise BEV representations. Following this, a Temporal and Deformable Cross-Fusion (TDCF) layer is utilized to encode multi-modal feature maps into a unified space-time dimension representation. Extensive experiments conducted on the nuScenes dataset show that FDARC achieves state-of-the-art performance in 3D detection tasks, markedly outperforming baseline models on the nuScenes val set using a ResNet-50 backbone, which attains 53.5% nuScenes Detection Score (NDS) and 44.7% mean Average Precision (mAP).

Links and sources

• PDF from original source

Comments