Hua, H, Li, D, Li, R et al. (3 more authors) (2022) Towards Explainable Action Recognition by Salient Qualitative Spatial Object Relation Chains. In: Proceedings of the 36th AAAI Conference on Artificial Intelligence: AAAI-22 Technical Tracks 5. Thirty-Sixth AAAI Conference on Artificial Intelligence, 22 Feb - 01 Mar 2022, Virtual. AAAI Press , Palo Alto, California USA , pp. 5710-5718. ISBN 978-1-57735-876-3
Abstract
In order to be trusted by humans, Artificial Intelligence agents should be able to describe rationales behind their decisions. One such application is human action recognition in critical or sensitive scenarios, where trustworthy and explainable action recognizers are expected. For example, reliable pedestrian action recognition is essential for self-driving cars and explanations for real-time decision making are critical for investigations if an accident happens. In this regard, learning-based approaches, despite their popularity and accuracy, are disadvantageous due to their limited interpretability. This paper presents a novel neuro-symbolic approach that recognizes actions from videos with human-understandable explanations. Specifically, we first propose to represent videos symbolically by qualitative spatial relations between objects called qualitative spatial object relation chains. We further develop a neural saliency estimator to capture the correlation between such object relation chains and the occurrence of actions. Given an unseen video, this neural saliency estimator is able to tell which object relation chains are more important for the action recognized. We evaluate our approach on two real-life video datasets, with respect to recognition accuracy and the quality of generated action explanations. Experiments show that our approach achieves superior performance on both aspects to previous symbolic approaches, thus facilitating trustworthy intelligent decision making. Our approach can be used to augment state-of-the-art learning approaches with explainabilities.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Keywords: | Knowledge Representation And Reasoning (KRR), Cognitive Modeling & Cognitive Systems (CMS), Computer Vision (CV), Intelligent Robotics (ROB) |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Engineering & Physical Sciences (Leeds) > School of Computing (Leeds) |
Depositing User: | Symplectic Publications |
Date Deposited: | 20 Apr 2022 12:47 |
Last Modified: | 17 Oct 2023 13:08 |
Status: | Published |
Publisher: | AAAI Press |
Identification Number: | 10.1609/aaai.v36i5.20513 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:185864 |