Alomari, M orcid.org/0000-0002-6565-4887, Li, F, Hogg, DC orcid.org/0000-0002-6125-9564 et al. (1 more author) (2022) Online perceptual learning and natural language acquisition for autonomous robots. Artificial Intelligence, 303. 103637. p. 103637. ISSN 0004-3702
Abstract
In this work, the problem of bootstrapping knowledge in language and vision for autonomous robots is addressed through novel techniques in grammar induction and word grounding to the perceptual world. In particular, we demonstrate a system, called OLAV, which is able, for the first time, to (1) learn to form discrete concepts from sensory data; (2) ground language (n-grams) to these concepts; (3) induce a grammar for the language being used to describe the perceptual world; and moreover to do all this incrementally, without storing all previous data. The learning is achieved in a loosely-supervised manner from raw linguistic and visual data. Moreover, the learnt model is transparent, rather than a black-box model and is thus open to human inspection. The visual data is collected using three different robotic platforms deployed in real-world and simulated environments and equipped with different sensing modalities, while the linguistic data is collected using online crowdsourcing tools and volunteers. The analysis performed on these robots demonstrates the effectiveness of the framework in learning visual concepts, language groundings and grammatical structure in these three online settings.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2021 Published by Elsevier B.V. This is an author produced version of a paper published in Artificial Intelligence. Uploaded in accordance with the publisher's self-archiving policy. This manuscript version is made available under the Creative Commons CC-BY-NC-ND 4.0 license http://creativecommons.org/licenses/by-nc-nd/4.0/ |
Keywords: | Language and vision; Language acquisition; Language grounding; Grammar induction |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Engineering & Physical Sciences (Leeds) > School of Computing (Leeds) |
Funding Information: | Funder Grant number EU - European Union FP7-ICT-600623 Alan Turing Institute No ref given |
Depositing User: | Symplectic Publications |
Date Deposited: | 01 Dec 2021 14:31 |
Last Modified: | 12 Dec 2024 15:24 |
Status: | Published |
Publisher: | Elsevier |
Identification Number: | 10.1016/j.artint.2021.103637 |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:181078 |
Download
Filename: Mo_AIJ_paper_final-accepted-version.pdf
Licence: CC-BY-NC-ND 4.0