Embodied Intelligence Security with Vision-language Models: A Survey

Engineering notes

Key topics: self-driving car, self-driving, perception. See the paper for implementation details and experimental results.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

Abstract Embodied intelligence (EI), integrating vision-language models (VLMs) with action-oriented capabilities, presents transformative potential for autonomous systems. However, deploying VLMs in safety-critical applications like self-driving cars and collaborative robots introduces significant challenges. Key concerns include adversarial attacks and content manipulation, through which maliciously altered visual or textual inputs could lead to incorrect interpretations and hazardous actions in EI systems. Furthermore, integrating VLMs into embodied agents amplifies these risks, as real-world physical interactions introduce complex safety-critical scenarios where errors in perception or decision-making can have immediate and severe consequences. While VLM integration amplifies safety concerns, these models simultaneously offer a foundation for defensive strategies aimed at enhancing the reliability of EI systems. Additionally, VLM-based EI systems raise critical ethical concerns regarding accountability, embedded biases and societal displacement. This review systematically analyzes current applications, safety risks, protection methods and ethical implications, while proposing research directions to advance trustworthy EI systems.

Links and sources

Comments