Systematic Situation Coverage versus Random Situation Coverage for Safety Testing in an Autonomous Car Simulation

Autonomous vehicles (AV) have the potential to improve road transport, but faults in the autonomous driving software can result in serious accidents. To assess the safety of AV driving software, we need to consider the wide variety and diversity of situations that it may encounter. Explicit situation coverage has previously been presented, but its usefulness has received a little empirical scrutiny. In this study, we evaluate a situation coverage based safety testing approach by comparing the performance of random and situation coverage-based test generation in terms of its ability to detect seeded faults in our ego AV at a road intersection under diverse environmental conditions. Our results suggest that this implementation of situation coverage, at least, does not provide an advantage over random generation.