Research on embodied agent multimodal perception and real-time path planning algorithms for complex unstructured environments

Engineering notes

Key topics: autonomous driving, end-to-end, path planning, lidar, deployment, perception, planning. See the paper for implementation details and experimental results.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

Autonomous navigation of embodied agents in complex unstructured environments demands tightly coupled multimodal perception and real-time path planning capabilities, forming a core technical bottleneck in physical-world robot deployment. Heterogeneous sensor data from visual, LiDAR, and depth modalities remain difficult to align and fuse under varying illumination and terrain conditions, while dynamic obstacle configurations impose severe latency constraints that existing planning algorithms fail to satisfy simultaneously. This paper proposes an integrated end-to-end framework combining a Cross-Modal Attention Fusion (CMAF) module, a Kalman-Graph Neural Network (K-GNN) dynamic obstacle predictor, and a two-layer Proximal Policy Optimization path planning architecture. The Cross-Modal Attention Fusion module fuses three-modal features through a multi-head attention mechanism, achieving a mean Intersection over Union of 78.6% with a fusion latency of 5.3 ms on a self-built unstructured environment dataset. The Kalman-Graph Neural Network couples Kalman filter physical motion priors with graph neural network interaction modeling to predict short-term trajectories of multiple moving obstacles online. The two-layer planner integrates fused perception features with a global semantic topology path to output local velocity commands in real time, reducing average planning time to 18.4 ms. Experiments on a Gazebo simulation platform and a self-developed four-wheeled robot across 60 unstructured test cases demonstrate a navigation success rate of 94.5%, surpassing the strongest baseline by 7.8 percentage points and satisfying real-time operational requirements.

Links and sources

• PDF from original source

Comments