VDNeRF: Vision‐Only Dynamic Neural Radiance Field for Urban Scenes

Engineering notes

Key topics: autonomous driving, perception. See the paper for implementation details and experimental results.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

ABSTRACT Neural radiance fields (NeRFs) implicitly model continuous three‐dimensional scenes using a set of images with known camera poses, enabling the rendering of photorealistic novel views. However, existing NeRF‐based methods encounter challenges in applications such as autonomous driving and robotic perception, primarily due to the difficulty of capturing accurate camera poses and limitations in handling large‐scale dynamic environments. To address these issues, we propose vision‐only dynamic NeRF (VDNeRF), a method that accurately recovers camera trajectories and learns spatiotemporal representations for dynamic urban scenes without requiring additional camera pose information or expensive sensor data. VDNeRF employs two separate NeRF models to jointly reconstruct the scene. The static NeRF model optimises camera poses and static background, whereas the dynamic NeRF model incorporates the 3D scene flow to ensure accurate and consistent reconstruction of dynamic objects. To address the ambiguity between camera motion and independent object motion, we design an effective and powerful training framework to achieve robust camera pose estimation and self‐supervised decomposition of static and dynamic elements in a scene. Extensive evaluations on mainstream urban driving datasets demonstrate that VDNeRF surpasses state‐of‐the‐art NeRF‐based pose‐free methods in both camera pose estimation and dynamic novel view synthesis.

Links and sources

Comments