Cross, M. and Ragni, A. orcid.org/0000-0003-0634-4456 (2024) What happens to diffusion model likelihood when your model is conditional? In: Coelho, C., Zimmering, B., Fernanda, M., Costa, P., Ferras, L.L. and Niggemann, O., (eds.) Proceedings of Machine Learning Research. 1st ECAI Workshop on “Machine Learning Meets Differential Equations: From Theory to Applications”, 20 Oct 2024, Santiago de Compostela, Spain. Proceedings of Machine Learning Research , pp. 1-14.
Abstract
Diffusion Models (DMs) iteratively denoise random samples to produce high-quality data. The iterative sampling process is derived from Stochastic Differential Equations (SDEs), allowing a speed-quality trade-off chosen at inference. Another advantage of sampling with differential equations is exact likelihood computation. These likelihoods have been used to rank unconditional DMs and for out-of-domain classification. Despite the many existing and possible uses of DM likelihoods, the distinct properties captured are unknown, especially in conditional contexts such as Text-To-Image (TTI) or Text-To-Speech synthesis (TTS). Surprisingly, we find that TTS DM likelihoods are agnostic to the text input. TTI likelihood is more expressive but cannot discern confounding prompts. Our results show that applying DMs to conditional tasks reveals inconsistencies and strengthens claims that the properties of DM likelihood are unknown. This impact sheds light on the previously unknown nature of DM likelihoods. Although conditional DMs maximise likelihood, the likelihood in question is not as sensitive to the conditioning input as one expects. This investigation provides a new point-of-view on diffusion likelihoods.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Editors: |
|
Copyright, Publisher and Additional Information: | © The authors 2024. Except as otherwise noted, this paper published in Proceedings of Machine Learning Research is made available under the terms of the Creative Commons Attribution 4.0 International License (CC-BY 4.0), which permits unrestricted use, distribution and reproduction in any medium, provided the original work is properly cited. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ |
Keywords: | Diffusion models; score-based generative modelling; likelihood |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 23 Jan 2025 11:22 |
Last Modified: | 28 Jan 2025 15:08 |
Published Version: | https://proceedings.mlr.press/v255/cross24a.html |
Status: | Published |
Publisher: | Proceedings of Machine Learning Research |
Refereed: | Yes |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:221602 |