Yang, Jingbo and Bors, Adrian Gheorghe orcid.org/0000-0001-7838-0021 (2023) Enabling the Encoder-Empowered GAN-based Video Generators for Long Video Generation. In: IEEE International Conference on Image Processing (ICIP). IEEE , Kuala Lumpur, Malayesia , pp. 1425-1429.
Abstract
Despite the remarkable progress in the video generation field, generating videos of longer-term remains challenging due to the challenge of sustaining the temporal consistency and continuity in the resulting synthesized movement while ensuring realism. In this paper, we propose a recall mechanism for enabling an encoder-empowered short-term video generator to produce long-term videos. This mechanism connects smoothly short video clips by modeling their temporal connections. We propose the Recall Encoder-GAN3 (REncGAN3), which enables an Encoder-based GenerativeAdversarial Network (GAN) to connect short generated videoclips into longer sequences of hundreds of frames. The recall mechanism, defined through a loss function, enables an appropriate plasticity continuity balance in the resulting long video stream. The proposed long-term video generation method ensures the generation of several hundred frames displaying consistent movement, which is non-repetitive while the computational memory costs are similar to those of short video generation models.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | ©IEEE 2023. This is an author-produced version of the published paper. Uploaded in accordance with the University’s Research Publications and Open Access policy. |
Dates: |
|
Institution: | The University of York |
Academic Units: | The University of York > Faculty of Sciences (York) > Computer Science (York) |
Depositing User: | Pure (York) |
Date Deposited: | 20 Dec 2024 12:30 |
Last Modified: | 21 Dec 2024 00:06 |
Published Version: | https://doi.org/10.1109/ICIP49359.2023.10222725 |
Status: | Published |
Publisher: | IEEE |
Identification Number: | 10.1109/ICIP49359.2023.10222725 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:221035 |