Peng, Z, Wang, H orcid.org/0000-0002-2281-5679, Yang, Y et al. (2 more authors) (Cover date: September 2023) Unsupervised image translation with distributional semantics awareness. Computational Visual Media, 9 (3). pp. 619-631. ISSN 2096-0433
Abstract
Unsupervised image translation (UIT) studies the mapping between two image domains. Since such mappings are under-constrained, existing research has pursued various desirable properties such as distributional matching or two-way consistency. In this paper, we re-examine UIT from a new perspective: distributional semantics consistency, based on the observation that data variations contain semantics, e.g., shoes varying in colors. Further, the semantics can be multi-dimensional, e.g., shoes also varying in style, functionality, etc. Given two image domains, matching these semantic dimensions during UIT will produce mappings with explicable correspondences, which has not been investigated previously. We propose distributional semantics mapping (DSM), the first UIT method which explicitly matches semantics between two domains. We show that distributional semantics has been rarely considered within and beyond UIT, even though it is a common problem in deep learning. We evaluate DSM on several benchmark datasets, demonstrating its general ability to capture distributional semantics. Extensive comparisons show that DSM not only produces explicable mappings, but also improves image quality in general.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © The Author(s) 2023. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creativecommons.org/licenses/by/4.0/. |
Keywords: | generative adversarial networks (GANs); manifold alignment; unsupervised learning; image-to-image translation; distributional semantics |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Engineering & Physical Sciences (Leeds) > School of Computing (Leeds) |
Depositing User: | Symplectic Publications |
Date Deposited: | 17 May 2022 12:28 |
Last Modified: | 18 Aug 2023 15:18 |
Published Version: | https://link.springer.com/article/10.1007/s41095-0... |
Status: | Published |
Publisher: | SpringerOpen |
Identification Number: | 10.1007/s41095-022-0295-3 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:186930 |
Download
Filename: Unsupervised image translation with distributional semantics awareness.pdf
Licence: CC-BY 4.0