DRIVE‐SAFE: Data‐Driven Robustness and Informed Validation for Evolving Specifications via Formal Evaluation

Engineering notes

Key topics: autonomous driving. See the paper for implementation details and experimental results.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

We present a regulator‐driven framework for the temporal evaluation of learning‐based, black‐box autonomous robot policies under evolving safety requirements. Motivated by real‐world certification processes in which regulators often assess observable behavior in inaccessible black‐box policies, we simulate a scenario where deployed policies must maintain ongoing compliance as human‐defined safety rules change over time. Natural‐language safety requirements are translated into Signal Temporal Logic (STL) specifications, which are then used to quantitatively evaluate rollout traces from black‐box policies. We compute Total Robustness Value (TRV), Largest Robustness Value (LRV), and Average Violation Robustness Voting (AVRV), to summarize average performance, worst‐case violation, and average specification violation across a set of rollout trajectories. These metrics are combined with domain‐informed importance weights to provide targeted feedback to model designers, indicating where retraining or redesign is most needed. In a virtual driving scenario, iterative retraining increased satisfaction of a global speed limit specification by 72% and a slippage mitigation specification by 32%. When the speed‐limit requirement was tightened, the resulting retrained model increased satisfaction by 47% but revealed trade‐offs with competing safety objectives. Overall, our approach integrates temporal‐logic‐based robustness metrics with statistically meaningful evaluation, bridging regulator oversight and model improvement under evolving safety requirements.

Links and sources

Comments