Al-Omari, M, Duckworth, P orcid.org/0000-0001-9052-6919, Hogg, DC orcid.org/0000-0002-6125-9564 et al. (1 more author) (2017) Natural Language Acquisition and Grounding for Embodied Robotic Systems. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. Thirty-First AAAI Conference on Artificial Intelligence, 04-09 Feb 2017, San Francisco, CA, USA. Association for the Advancement of Artificial Intelligence , pp. 4349-4356.
Abstract
We present a cognitively plausible novel framework capable of learning the grounding in visual semantics and the grammar of natural language commands given to a robot in a table top environment. The input to the system consists of video clips of a manually controlled robot arm, paired with natural language commands describing the action. No prior knowledge is assumed about the meaning of words, or the structure of the language, except that there are different classes of words (corresponding to observable actions, spatial relations, and objects and their observable properties). The learning process automatically clusters the continuous perceptual spaces into concepts corresponding to linguistic input. A novel relational graph representation is used to build connections between language and vision. As well as the grounding of language to perception, the system also induces a set of probabilistic grammar rules. The knowledge learned is used to parse new commands involving previously unseen objects.
Metadata
Authors/Creators: |
|
||||
---|---|---|---|---|---|
Keywords: | Language and Vision; Cognitive Robotics | ||||
Dates: |
|
||||
Institution: | The University of Leeds | ||||
Academic Units: | The University of Leeds > Faculty of Engineering & Physical Sciences (Leeds) > School of Computing (Leeds) The University of Leeds > Faculty of Engineering & Physical Sciences (Leeds) |
||||
Funding Information: |
|
||||
Depositing User: | Symplectic Publications | ||||
Date Deposited: | 07 Dec 2016 14:37 | ||||
Last Modified: | 20 Jun 2021 08:37 | ||||
Published Version: | https://ojs.aaai.org/index.php/AAAI/article/view/1... | ||||
Status: | Published | ||||
Publisher: | Association for the Advancement of Artificial Intelligence | ||||
Related URLs: |