Alomari, M orcid.org/0000-0002-6565-4887, Duckworth, P orcid.org/0000-0001-9052-6919
, Hogg, DC orcid.org/0000-0002-6125-9564
et al. (1 more author)
(2017)
Natural Language Acquisition and Grounding for Embodied Robotic Systems.
In:
Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence.
Thirty-First AAAI Conference on Artificial Intelligence, 04-09 Feb 2017, San Francisco, CA, USA.
Association for the Advancement of Artificial Intelligence
, pp. 4349-4356.
Abstract
We present a cognitively plausible novel framework capable of learning the grounding in visual semantics and the grammar of natural language commands given to a robot in a table top environment. The input to the system consists of video clips of a manually controlled robot arm, paired with natural language commands describing the action. No prior knowledge is assumed about the meaning of words, or the structure of the language, except that there are different classes of words (corresponding to observable actions, spatial relations, and objects and their observable properties). The learning process automatically clusters the continuous perceptual spaces into concepts corresponding to linguistic input. A novel relational graph representation is used to build connections between language and vision. As well as the grounding of language to perception, the system also induces a set of probabilistic grammar rules. The knowledge learned is used to parse new commands involving previously unseen objects.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Keywords: | Language and Vision; Cognitive Robotics |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Engineering & Physical Sciences (Leeds) > School of Computing (Leeds) The University of Leeds > Faculty of Engineering & Physical Sciences (Leeds) |
Funding Information: | Funder Grant number EU - European Union FP7-ICT-600623 |
Depositing User: | Symplectic Publications |
Date Deposited: | 07 Dec 2016 14:37 |
Last Modified: | 12 Dec 2024 15:23 |
Published Version: | https://ojs.aaai.org/index.php/AAAI/article/view/1... |
Status: | Published |
Publisher: | Association for the Advancement of Artificial Intelligence |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:109107 |