Zhang, G. orcid.org/0009-0009-7811-065X, Wang, S. orcid.org/0000-0001-5620-9151, Xie, Y. orcid.org/0000-0003-1158-1587 et al. (3 more authors) (2024) A Task-Oriented Grasping Framework Guided by Visual Semantics for Mobile Manipulators. IEEE Transactions on Instrumentation and Measurement, 73. 7504213. ISSN 0018-9456
Abstract
The densely cluttered operational environment and the absence of object information hinder mobile manipulators from achieving specific grasping tasks. To address this issue, this article proposes a task-oriented grasping framework guided by visual semantics for mobile manipulators. With multiple attention mechanisms, we first present a modified DeepLabV3+ model by replacing the backbone networks with MobileNetV2 and incorporating a novel attention feature fusion module (AFFM) to build a preprocessing module, thus producing semantic images efficiently and accurately. A semantic-guided viewpoint adjustment strategy is designed in which the semantic images are used to calculate the optimal viewpoint that enables the eye-in-hand installed camera to self-adjust until it encompasses all the objects within the task-related area. Based on the improved DeepLabV3+ model and the generative residual convolutional neural network, a task-oriented grasp detection structure is developed to generate a more precise grasp representation for the specific object in densely cluttered scenarios. The effectiveness of the proposed framework is validated through the dataset comparison tests and multiple sets of practical grasping experiments. The results demonstrate that our proposed method achieves competitive results versus the state-of-the-art (SOTA) methods, which attains an accuracy of 98.3% on the Cornell grasping dataset and achieves a grasping success rate of 91% in densely cluttered scenes.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: | |
Copyright, Publisher and Additional Information: | © 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. |
Keywords: | Absence of object information, deep learning, mobile manipulator, task-oriented robotic grasping, visual semantics |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Engineering & Physical Sciences (Leeds) > School of Electronic & Electrical Engineering (Leeds) > Robotics, Autonomous Systems & Sensing (Leeds) |
Depositing User: | Symplectic Publications |
Date Deposited: | 03 May 2024 09:27 |
Last Modified: | 03 May 2024 09:27 |
Status: | Published |
Publisher: | Institute of Electrical and Electronics Engineers (IEEE) |
Identification Number: | 10.1109/tim.2024.3381662 |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:212204 |