Effective and reflective assurance for AI-based autonomy

This paper outlines the challenges in assuring the safety of autonomous AI-based systems and the ensuing gaps in assurance that need to be closed to achieve a comparable level of integrity as for conventional safety-critical systems. By analysing the underlying causes of uncertainty in assurance arguments for these systems, as well as current research on this topic, the paper identifies extensions to conventional safety assurance approaches, that, if combined within a holistic framework, may go some way to reducing these gaps. As part of this approach, we recommend the use of causal models of the environment and system that enable more rigorous statements to be made about the completeness and consistency of safety requirements and for supporting verification and validation activities. Nevertheless, some assurance uncertainty will inevitably remain as an inherent consequence of the environmental and system complexity. Therefore, a reflective and iterative approach to assurance is deemed essential to create transparency regarding the strength or otherwise of the safety claims that can be made about the system.