Zhao, C., Sun, L. orcid.org/0000-0002-0393-8665 and Stolkin, R. (2020) Simultaneous material segmentation and 3D reconstruction in industrial scenarios. Frontiers in Robotics and AI, 7. 52. ISSN 2296-9144
Abstract
Recognizing material categories is one of the core challenges in robotic nuclear waste decommissioning. All nuclear waste should be sorted and segregated according to its materials, and then different disposal post-process can be applied. In this paper, we propose a novel transfer learning approach to learn boundary-aware material segmentation from a meta-dataset and weakly annotated data. The proposed method is data-efficient, leveraging a publically available dataset for general computer vision tasks and coarsely labeled material recognition data, with only a limited number of fine pixel-wise annotations required. Importantly, our approach is integrated with a Simultaneous Localization and Mapping (SLAM) system to fuse the per-frame understanding delicately into a 3D global semantic map to facilitate robot manipulation in self-occluded object heaps or robot navigation in disaster zones. We evaluate the proposed method on the Materials in Context dataset over 23 categories and that our integrated system delivers quasi-real-time 3D semantic mapping with high-resolution images. The trained model is also verified in an industrial environment as part of the EU RoMaNs project, and promising qualitative results are presented. A video demo and the newly generated data can be found at the project website.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2020 Zhao, Sun and Stolkin. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). (http://creativecommons.org/licenses/by/4.0/) The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
Keywords: | material segmentation; 3D material reconstruction; transfer learning; deep neural network; nuclear applications |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
Funding Information: | Funder Grant number EU Horizon 2020 645582 ENGINEERING AND PHYSICAL SCIENCE RESEARCH COUNCIL EP/R026092/1 |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 29 May 2020 16:27 |
Last Modified: | 29 May 2020 16:29 |
Status: | Published |
Publisher: | Frontiers Media SA |
Refereed: | Yes |
Identification Number: | 10.3389/frobt.2020.00052 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:161166 |