NMR: Neural Manifold Representation for Autonomous Driving

Engineering notes

Key topics: autonomous driving, bev, end-to-end, path planning, occupancy, carla, perception, prediction, planning. See the paper for implementation details and experimental results.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

Autonomous driving requires efficient reasoning about the Spatio-temporal nature of the semantics of the scene. Recent ap-proaches have successfully amalgamated the traditional modular architecture of an autonomous driving stack comprising perception, prediction, and planning in an end-to-end trainable system. Such a system calls for a shared latent space embedding with interpretable intermediate trainable projected representation. One such success- fully deployed representation is the Bird’s-Eye View(BEV) representation of the scene in ego-frame. However, a fundamental assump- tion for an undistorted BEV is the local coplanarity of the world around the ego-vehicle. This assumption is highly restrictive, as roads, in general, do have gradients. The resulting distortions make path planning inefficient and incorrect. To overcome this limitation, we propose Neural Manifold Representation (NMR), a representation for the task of autonomous driving that learns to infer semantics and predict way-points on a manifold over a finite horizon, centered on the ego-vehicle. We do this using an iterative attention mecha- nism applied on a latent high dimensional embedding of surround monocular images and partial ego-vehicle state. This representa- tion helps generate motion and behavior plans consistent with and cognizant of the surface geometry. We propose a sampling algo- rithm based on edge-adaptive coverage loss of BEV occupancy grid and associated guidance flow field to generate the surface manifold while incurring minimal computational overhead. We aim to test the efficacy of our approach on CARLA and SYNTHIA-SF.

Links and sources

• Official / arXiv page
• PDF from original source

Comments