Towards Self-Aware Autonomous Agents: A Multi-Model Sensing and Communication Framework

Engineering notes

Comparative evaluations against deep learning baselines such as convolutional neural networks (CNNs) and long short-term memory (LSTM) networks demonstrate superior prediction accuracy.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

Intelligent transportation systems (ITSs) are reshaping modern mobility by enabling real-time communication, situational awareness, and coordinated decision-making across vehicles and infrastructure. Central to these systems is vehicle-to-everything (V2X) communication relying on uninterrupted data exchange between the various entities of the environment to support safety, sustainable efficiency for connected autonomous vehicles (AVs). The integration of V2X communication paradigms with sixth-generation (6G) and beyond wireless communication technologies, combined with artificial intelligence (AI) framework facilitates ultra reliable and low latency communication, which is essential for real-time decision making for AVs and smart cities. The sensors connected to AVs, proprioceptive and exteroceptive, enable them to perceive both their internal state and external environment, supporting rapid responses to critical events. Integrated sensing and communication (ISAC) enhances these capabilities by combining perception and communication, allowing V2X systems to adapt intelligently to real-time emergencies for safe and efficient operation of autonomous driving systems. However, ISAC remains vulnerable to dynamic blockages in complex environments. This thesis presents a multi-stage framework primarily aiming at detecting various blockages, including long-term, short-term, and unknown sensor blockages, as well as line-of-sight (LOS) link blockages. Considering the challenges in current AI approaches, especially their black-box nature and limited performance under new experiences, the work presented in this thesis is motivated by human-inspired reasoning and learning models that propagate messages and update beliefs over time to overcome these limitations. Therefore, this work employs data-driven, probabilistic, and hierarchical reasoning techniques based on real-time sensor data to enhance the self-awareness capabilities of autonomous agents. Initially, dynamic probabilistic models are leveraged to detect anomalies when sensor observations are unavailable due to harsh weather conditions. Taking advantage of the hierarchical nature of these models, interaction rules between various vehicles are encoded in dictionaries, and the concept of dummy nodes is introduced and LiDAR sensor blockages are inferred from temporal LiDAR observations. In later stages, LiDAR and RF power measurements are jointly utilized under an ISAC paradigm to infer blockages and detect abnormalities. Comparative evaluations against deep learning baselines such as convolutional neural networks (CNNs) and long short-term memory (LSTM) networks demonstrate superior prediction accuracy. The proposed models provide explanations when the environment changes at the observation layer. Hence, the findings contribute to the development of explainable autonomous agents, advancing the state of self-aware autonomous systems in uncertain environments and paving the way toward connected and safe V2X communication by improving latency and reliability in blockage-prone environments.

Links and sources

Comments