Aligning LiDAR to Vision Language Space: A Fusion Approach for Steering Angle Regression of a Vehicle

Engineering notes

Our code is available at: https://github.com/ParvezAlam123/Aligning-LiDAR-to-Vision-Language-Space-A-fusion-approach-for-Steering-Angle-Regression-of-a-Vehicle

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

Steering angle prediction plays an important role in autonomous vehicle control. Existing approaches rely on single modality like camera or LiDAR data to predict the steering angle, ignoring the rich information obtained after fusing all sensor modalities. In this work we proposed the multimodal fusion framework to improve the steering angle prediction accuracy. Specifically, we aligned LiDAR features to the FLAVA vision language space and fused these aligned features with image features to predict the steering angle of the vehicle. The proposed method demonstrates the considerable accuracy in steering angle prediction outperforming single modality approaches and highlighting the importance of multimodal fusion to predict the steering angle of the vehicle. Our code is available at: https://github.com/ParvezAlam123/Aligning-LiDAR-to-Vision-Language-Space-A-fusion-approach-for-Steering-Angle-Regression-of-a-Vehicle

Links and sources

• Official / arXiv page

Comments