PWCC: Replication Package for Probability-Weighted Combinatorial Coverage

Engineering notes

Key topics: autonomous driving system, autonomous driving, self-driving car, self-driving, autonomous vehicle. See the paper for implementation details and experimental results.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为端到端自动驾驶、BEV感知、3D目标检测、轨迹预测、路径规划、LiDAR感知等高价值论文补充中文说明。

Original abstract

Replication package for the paper "PWCC: Probability-Weighted Combinatorial Coverage for Risk-Informed Scenario Testing of Autonomous Systems" by Borhaneddine Hamadou, Chokri Mraidha, and Nikolaos Matragkas, published at the 41st IEEE/ACM International Conference on Automated Software Engineering (ASE '26), October 12-16, 2026, Munich, Germany. DOI: 10.1145/3832783.3837443 PWCC is a test adequacy metric for scenario-based testing of autonomous systems. It learns a Bayesian network model of expected operating conditions from operational profiling data, assigns credited probability mass to t-way combinations of scenario features, and declares testing adequate when the uncovered mass falls below a risk-calibrated threshold, providing a principled stopping criterion for test campaigns. CONTENTS The PWCC implementation: coverage metric, minimum-exposure criterion, and adequacy rule pwcc_tool.py, a command-line tool that applies PWCC to your own data. Given operational profiling data and a set of executed scenarios, it learns the ODD model, returns an adequacy verdict, and reports which feature combinations remain under-tested and how much additional exposure each requires. It exits non-zero when testing is inadequate, so it can be used directly as a CI stopping gate. Evaluation scripts reproducing all five research questions (RQ1-RQ5), the seven baselines, and the ablations Pre-computed results for every research question, so all paper figures and tables regenerate in about 30 seconds without re-running the pipeline and without MATLAB (for TISA) The four evaluation datasets in profiling and validation splits: AsFault, DeepScenario, DeepCollision, and Frost FILES IN THIS RECORD pwcc-artifact-v3.zip: the complete replication package pwcc-v3-docker.tar.gz: a pre-built container image (linux/amd64) with every dependency installed GETTING STARTED Using the pre-built image, no build required: docker load -i pwcc-v3-docker.tar.gz docker run --rm pwcc:v3 ./reproduce.sh smoke # 31 tests, ~1 min docker run --rm pwcc:v3 ./reproduce.sh figures # all paper figures and tables, ~30 s docker run --rm pwcc:v3 python pwcc_tool.py --help Alternatively, install natively with Python 3.12 and pip install -r requirements.txt. See README.md for full reproduction instructions, REQUIREMENTS.md for environment details, and data/PROVENANCE.md for dataset documentation. LICENSING The PWCC implementation, evaluation scripts, and the Frost dataset are released under CC-BY-4.0. Frost is derived from meteorological observations published by the Norwegian Meteorological Institute (MET Norway) under CC-BY-4.0; MET Norway is credited as the source of the underlying observations. The AsFault, DeepScenario, and DeepCollision datasets are redistributed under their original licenses and remain the property of their respective authors; see the corresponding publications for terms and attribution: AsFault: Gambi, Mueller, Fraser. Automatically testing self-driving cars with search-based procedural content generation. ISSTA 2019. DeepScenario: Lu, Yue, Ali. DeepScenario: An open driving scenario dataset for autonomous driving system testing. MSR 2023. DeepCollision: Lu et al. Learning configurations of operating environment of autonomous vehicles to maximize their collisions. TSE 2022. The TISA baseline is redistributed from the replication package of Neelofar & Aleti, Towards Reliable AI: Adequacy Metrics for Ensuring the Quality of System-level Testing of Autonomous Vehicles, ICSE 2024 (doi:10.1145/3597503.3623314; package doi:10.5281/zenodo.7784015, CC-BY-4.0). It bundles the Instance Space Analysis (MATILDA) MATLAB toolkit, which is licensed under GPL-3.0 and retains that license here as an unmodified separate work. Full details are in LICENSE.

Links and sources

Comments