Object localization in images via fusion of SFM and YOLO

Engineering notes

Key topics: autonomous driving, object detection, point cloud. See the paper for implementation details and experimental results.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

<title>Abstract</title> Along with the rapid development of computer vision, unmanned aerial vehicle (UAV) technology, and autonomous driving, the precise acquisition of coordinate information for captured objects in geospatial space has become a core requirement for numerous applications. Traditional object detection methods, such as the YOLO series algorithms, primarily focus on object categories and two-dimensional bounding boxes, lacking the capability for precise estimation of object spatial positions. In this paper, a novel framework fusing the Structure from Motion (SfM) algorithm with an improved YOLOv8 object detection model is proposed to achieve high-precision geospatial localization of objects of interest in image sequences. Initially, the SfM algorithm is utilized to perform three-dimensional reconstruction on multi-view images, generating sparse point clouds with spatial coordinates. Simultaneously, a YOLOv8 model incorporating a hybrid attention mechanism (GAM and CA) is employed for precise detection of target objects. Finally, by reprojecting the 3D point cloud to the image plane and performing spatial constraint matching with two-dimensional detection boxes, a mapping relationship between two-dimensional detection results and three-dimensional spatial coordinates is established. Experimental results demonstrate that ablation studies on the VisDrone dataset confirm the effectiveness of the hybrid attention mechanism. Localization experiments on vehicle image sequences collected by UAVs verify that the proposed method can accurately acquire the geographical coordinates of captured objects. This study not only expands the functional boundaries of traditional object detection but also provides technical support for application scenarios such as UAV visual localization and intelligent traffic monitoring.

Links and sources

Comments