A Sensor Fusion System with Thermal Infrared Camera and LiDAR for Autonomous Vehicles: Its Calibration and Application

Engineering notes

Key topics: autonomous driving, autonomous vehicle, object detection, lidar, sensor fusion, radar, perception. See the paper for implementation details and experimental results.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

Vision, Radar, and LiDAR sensors are widely used for autonomous vehicle perception technology. Especially object detection and classification are primarily dependent on vision sensors. However, under poor lighting conditions, dazzling sunlight, or bad weathers an object might be difficult to be identified with general vision sensors. In this paper, we propose a sensor fusion system with a thermal infrared camera and LiDAR sensor that can reliably detect and identify objects even in environments where visibility is poor, such as in severe glare and fog or smoke. The proposed method obtains intrinsic parameters by calibrating the thermal infrared camera and LiDAR sensor. Extrinsic calibration algorithm between two sensors is made to obtain the extrinsic parameters (rotation and translation matrix) using 3D calibration targets. This system and proposed algorithm show that it can reliably detect and identify objects even in hard visibility environments, such as in severe glare due to direct sunlight or headlights or in low visibility environments, such as in severe fog or smoke.

Links and sources

• Official / arXiv page

Comments