Alomari, M orcid.org/0000-0002-6565-4887, Chinellato, E, Gatsoulis, Y et al. (2 more authors) (2016) Unsupervised Grounding of Textual Descriptions of Object Features and Actions in Video. In: Proceedings, 15th International Conference on Principles of Knowledge Representation and Reasoning. KR 2016, 25-29 Apr 2016, Cape Town, South Africa. Association for the Advancement of Artificial Intelligence , pp. 505-508.
Abstract
We propose a novel method for learning visual concepts and their correspondence to the words of a natural language. The concepts and correspondences are jointly inferred from video clips depicting simple actions involving multiple objects, together with corresponding natural language commands that would elicit these actions. Individual objects are first detected, together with quantitative measurements of their colour, shape, location and motion. Visual concepts emerge from the co-occurrence of regions within a measurement space and words of the language. The method is evaluated on a set of videos generated automatically using computer graphics from a database of initial and goal configurations of objects. Each video is annotated with multiple commands in natural language obtained from human annotators using crowd sourcing.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2016, Association for the Advancement of Artificial Intelligence. This is an author produced version of a paper published in Proceedings, 15th International Conference on Principles of Knowledge Representation and Reasoning (KR 2016). |
Keywords: | Grounding Symbols in Video, Aligning Language and Vision, Semantics of Relations |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Engineering & Physical Sciences (Leeds) > School of Computing (Leeds) > Artificial Intelligence & Biological Systems (Leeds) |
Funding Information: | Funder Grant number EU - European Union FP7-ICT-600623 |
Depositing User: | Symplectic Publications |
Date Deposited: | 04 Mar 2016 13:21 |
Last Modified: | 12 Dec 2024 12:40 |
Published Version: | http://www.aaai.org/ocs/index.php/KR/KR16/schedCon... |
Status: | Published |
Publisher: | Association for the Advancement of Artificial Intelligence |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:95572 |