Towards Robustness and Security of AI Systems through Reinforcement Learning-Based Analysis and Exploration

Engineering notes

This includes SafeAudit, a meta-audit framework for evaluating the completeness of agent tool-call safety benchmarks. SafeAudit systematically enumerates valid tool-call workflows and diverse user scenarios, then introduces rule-resistance as a quantitative proxy for identifying unsafe interaction patterns that remain uncovered by existing benchmark-derived safety rules.Together, these projects provide a unified view of AI security as an active exploration problem.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

Artificial intelligence (AI) systems are rapidly evolving from passive predictors into autonomous decision-makers that perceive, reason, and act across complex environments. This shift brings substantial benefits, but also expands the security surface from isolated model errors to sequential, interactive, and tool-mediated failures. In reinforcement learning (RL) systems, hidden backdoors can remain dormant until a trigger causes unsafe behavior. In large language models (LLMs), adversarial prompts can bypass alignment safeguards through adaptive exploration. In tool-using agents, safety increasingly depends not only on what the model says, but on which tools it calls, in what order, with what parameters, and under what contextual assumptions. As a result, trustworthy AI requires more than static evaluation or manual red-teaming; it requires systematic methods that can actively discover, understand, and mitigate vulnerabilities before deployment.This dissertation addresses this challenge through RL-based analysis and exploration for the robustness and security of AI systems. It studies AI security across three connected stages. First, it secures RL agents by developing attack-agnostic methods for detecting and removing backdoors in deep RL policies, and by exposing more realistic trajectory-level backdoor threats in end-to-end autonomous driving, where coordinated multi-vehicle behaviors can serve as physically plausible triggers. Second, it uses RL as a guided search engine for red-teaming LLMs, formulating jailbreaking prompts discovery as a black-box sequential decision-making problem and improving the efficiency of finding prompts that bypass model safeguards. Third, it investigates the safety of LLM agents, where model behavior is intertwined with tool use, memory, and real-world actions. This includes SafeAudit, a meta-audit framework for evaluating the completeness of agent tool-call safety benchmarks. SafeAudit systematically enumerates valid tool-call workflows and diverse user scenarios, then introduces rule-resistance as a quantitative proxy for identifying unsafe interaction patterns that remain uncovered by existing benchmark-derived safety rules.Together, these projects provide a unified view of AI security as an active exploration problem. Rather than treating safety evaluation as a fixed set of test cases, this dissertation develops methods that search for hidden failures, characterize their mechanisms, and design defenses or audits that improve system reliability. The resulting framework spans attack, defense, and evaluation across RL agents, LLMs, and tool-using agents, contributing practical techniques for building autonomous AI systems that remain robust as they gain greater agency and interact more directly with the real world.

Links and sources

Comments