Alomari, M orcid.org/0000-0002-6565-4887, Duckworth, P, Hawasly, M et al. (2 more authors) (2017) Natural Language Grounding and Grammar Induction for Robotic Manipulation Commands. In: Proceedings of the First Workshop on Language Grounding for Robotics. 1st Workshop on Language Grounding for Robotics at ACL 2017, 30 Jul - 04 Aug 2017, Vancouver, BC. The Association for Computational Linguistics , pp. 35-43. ISBN 978-1-945626-64-7
Abstract
We present a cognitively plausible system capable of acquiring knowledge in language and vision from pairs of short video clips and linguistic descriptions. The aim of this work is to teach a robot manipulator how to execute natural language commands by demonstration. This is achieved by first learning a set of visual `concepts' that abstract the visual feature spaces into concepts that have human-level meaning. Second, learning the mapping/grounding between words and the extracted visual concepts. Third, inducing grammar rules via a semantic representation known as Robot Control Language (RCL). We evaluate our approach against state-of-the-art supervised and unsupervised grounding and grammar induction systems, and show that a robot can learn to execute never seen-before commands from pairs of unlabelled linguistic and visual inputs.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Engineering & Physical Sciences (Leeds) > School of Computing (Leeds) |
Funding Information: | Funder Grant number EU - European Union FP7-ICT-600623 |
Depositing User: | Symplectic Publications |
Date Deposited: | 03 Aug 2017 11:21 |
Last Modified: | 12 Dec 2024 12:41 |
Published Version: | http://aclweb.org/anthology/W/W17/W17-28.pdf |
Status: | Published |
Publisher: | The Association for Computational Linguistics |
Identification Number: | 10.18653/v1/w17-2805 |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:119757 |