Jalal, M.A., Chen, R., Moore, R. et al. (1 more author) (2018) American sign language posture understanding with deep neural networks. In: 2018 21st International Conference on Information Fusion (FUSION). 2018 21st International Conference on Information Fusion (FUSION), 10-13 Jul 2018, Cambridge, UK. IEEE , UK , pp. 573-579. ISBN 978-0-9964527-6-2
Abstract
Sign language is a visually oriented, natural, nonverbal communication medium. Having shared similar linguistic properties with its respective spoken language, it consists of a set of gestures, postures and facial expressions. Though, sign language is a mode of communication between deaf people, most other people do not know sign language interpretations. Therefore, it would be constructive if we can translate the sign postures artificially. In this paper, a capsule-based deep neural network sign posture translator for an American Sign Language (ASL) fingerspelling (posture), has been presented. The performance validation shows that the approach can successfully identify sign language, with accuracy like 99%. Unlike previous neural network approaches, which mainly used fine-tuning and transfer learning from pre-trained models, the developed capsule network architecture does not require a pre-trained model. The framework uses a capsule network with adaptive pooling which is the key to its high accuracy. The framework is not limited to sign language understanding, but it has scope for non-verbal communication in Human-Robot Interaction (HRI) also.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2018 ISIF. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works. Reproduced in accordance with the publisher's self-archiving policy. |
Keywords: | American Sign Language (ASL) Understanding; Neural Network; Capsule Network; Adaptive Pooling |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Automatic Control and Systems Engineering (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 25 Sep 2018 08:44 |
Last Modified: | 25 Sep 2018 08:48 |
Published Version: | https://doi.org/10.23919/ICIF.2018.8455725 |
Status: | Published |
Publisher: | IEEE |
Refereed: | Yes |
Identification Number: | 10.23919/ICIF.2018.8455725 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:136087 |