M2FU: Multi-Modal Fusion for Urban Autonomous Driving

Engineering notes

The model was trained using an imitation learning approach and subsequently evaluated on the CARLA autonomous driving benchmark. The results demonstrate that the end-to-end model based on the dual attention transformer outperforms traditional classifier-based models in avoiding dynamic obstacles.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

Existing sensor fusion approaches are poor, this paper proposes a novel end-to-end autonomous driving model called M2FU. This model solves the problem of past end-to-end models that have many collisions, are heavily blocked, and are unable to effectively detect traffic lights. The perception module of the M2FU improves on the transformer-based detection model by using a dual-attention mechanism for its intermediate feature fusion and a recurrent neural network with gated recurrent units for path planning. In addition, it includes two controllers for managing the vehicle’s speed and steering angle. The model was trained using an imitation learning approach and subsequently evaluated on the CARLA autonomous driving benchmark. The results demonstrate that the end-to-end model based on the dual attention transformer outperforms traditional classifier-based models in avoiding dynamic obstacles. We validate the effectiveness of our method through experiments in urban environments with complex scenes, utilizing the CARLA urban driving simulator. Our driving scores improved by 17% compared to geometry-based fusion.

Links and sources

• Official / arXiv page

Comments