Zero-shot semantic landmark-based visual odometry using foundation models for unstructured planetary exploration

Engineering notes

Experimental results show that the proposed approach achieves a decimeter-level trajectory accuracy ( <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="m1"> <mml:mrow> <mml:mi>R</mml:mi> <mml:mi>M</mml:mi> <mml:mi>S</mml:mi> <mml:mi>E</mml:mi> <mml:mo>≈</mml:mo> </mml:mrow> </mml:math> 0.14 m) on the Martian analog and an <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="m2"> <mml:mrow> <mml:mi>R</mml:mi> <mml:mi>M</mml:mi> <mml:mi>S</mml:mi> <mml:mi>E</mml:mi> </mml:mrow> </mml:math> = 1.93 m on the most stable lunar traverse, without any domain-specific fine-tuning.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

Precise autonomous navigation on unstructured planetary surfaces is a critical prerequisite for future exploration missions, particularly in GNSS-denied environments such as the Lunar South Pole or Martian deserts. Traditional Visual Odometry (VO) methods, which rely on tracking low-level geometric features (e.g., corners), often fail under the extreme illumination contrast of the Moon or the textural monotony of the Martian regolith. In this work, we present a zero-shot semantic landmark-based visual odometry approach that leverages the generalization capabilities of modern Foundation Models. Our approach uses the Segment Anything Model (SAM) to extract geological landmarks (rocks) and DINOv2 to generate view-invariant semantic descriptors that are matched across frames. We evaluate our pipeline across two distinct domains: a high-fidelity synthetic lunar environment (LuSNAR dataset) to test robustness against extreme lighting, and a real-world Martian analog dataset (Katwijk Beach) to assess sim-to-real transfer. Experimental results show that the proposed approach achieves a decimeter-level trajectory accuracy ( <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="m1"> <mml:mrow> <mml:mi>R</mml:mi> <mml:mi>M</mml:mi> <mml:mi>S</mml:mi> <mml:mi>E</mml:mi> <mml:mo>≈</mml:mo> </mml:mrow> </mml:math> 0.14 m) on the Martian analog and an <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="m2"> <mml:mrow> <mml:mi>R</mml:mi> <mml:mi>M</mml:mi> <mml:mi>S</mml:mi> <mml:mi>E</mml:mi> </mml:mrow> </mml:math> = 1.93 m on the most stable lunar traverse, without any domain-specific fine-tuning. Our results suggest that Foundation-Model-based semantic landmarks are a promising alternative to low-level features for zero-shot VO in planetary-like environments.

Links and sources

• PDF from original source

Comments