Cao, G., Jiang, J., Mao, N. orcid.org/0000-0003-1203-9773 et al. (3 more authors) (2023) Vis2Hap: Vision-based Haptic Rendering by Cross-modal Generation. In: 2023 IEEE International Conference on Robotics and Automation (ICRA). 2023 IEEE International Conference on Robotics and Automation (ICRA), 29 May - 02 Jun 2023, London, UK. IEEE , pp. 12443-12449. ISBN 9798350323658
Abstract
To assist robots in teleoperation tasks, haptic rendering which allows human operators access a virtual touch feeling has been developed in recent years. Most previous haptic rendering methods strongly rely on data collected by tactile sensors. However, tactile data is not widely available for robots due to their limited reachable space and the restrictions of tactile sensors. To eliminate the need for tactile data, in this paper we propose a novel method named as Vis2Hap to generate haptic rendering from visual inputs that can be obtained from a distance without physical interaction. We take the surface texture of objects as key cues to be conveyed to the human operator. To this end, a generative model is designed to simulate the roughness and slipperiness of the object's surface. To embed haptic cues in Vis2Hap, we use height maps from tactile sensors and spectrograms from friction coefficients as the intermediate outputs of the generative model. Once Vis2Hap is trained, it can be used to generate height maps and spectrograms of new surface textures, from which a friction image can be obtained and displayed on a haptic display. The user study demonstrates that our proposed Vis2Hap method enables users to access a realistic haptic feeling similar to that of physical objects. The proposed vision-based haptic rendering has the potential to enhance human operators' perception of the remote environment and facilitate robotic manipulation.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Arts, Humanities and Cultures (Leeds) > School of Design (Leeds) |
Depositing User: | Symplectic Publications |
Date Deposited: | 11 Dec 2023 12:36 |
Last Modified: | 11 Dec 2023 12:36 |
Status: | Published |
Publisher: | IEEE |
Identification Number: | 10.1109/icra48891.2023.10160373 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:206418 |