Back to search

University of York

Long video generation using the VAE-GAN method

Abstract

dc:description.abstract

Video generation has emerged as a critical area in machine learning, with applications spanning entertainment, virtual reality, and surveillance. However, generating realistic and temporally coherent videos, especially for long-term sequences, remains challenging. This thesis addresses these challenges through novel hybrid models and transformer-based architectures, improving video quality, efficiency, and duration. The thesis first analyses limitations of existing generative models. While GANs produce sharp videos but suffer from computational expense and mode collapse, VAEs are more efficient but yield blurry outputs. We propose hybrid VAE-GAN models that combine their strengths by combining the inference ability of the VAE with the generative properties of the GAN, using a VAE encoder with GAN generators to enhance video consistency and continuity. Focusing on temporal modelling, we address the critical challenge of long-duration video generation under computational constraints. Emphasizing GPU memory efficiency, we develop a novel recall mechanism that decomposes videos into temporally coherent sub-sequences with Markovian dependencies. This enables efficient long-term modelling with fixed memory requirements. Further refinement through auto-regressive modelling enhances temporal consistency, while our introduction of the Generative Pre-trained Transformer (GPT) architecture provides global temporal perspective through latent space sequence modelling. The thesis provides several key contributions: (1) Encoding GAN3 (EncGAN3), integrating VAE and GAN for high-quality short-term videos; (2) Recall Encoding GAN3 (REncGAN) with a recall mechanism for efficient long-duration generation, developed through iterative architectural improvements; (3) Auto-Regressive R2 (AR2) with auto-regressive recall; and (4) GPT R2 (R3) leveraging transformer architectures. These models achieve minimal, fixed GPU memory increments regardless of video length. Experimental results demonstrate significant improvements in both short- and long-term video generation across multiple benchmarks, showing superior performance in quality, coherence, and computational efficiency. This thesis advances video generation techniques, particularly for applications requiring high-quality extended sequences.

Degree

thesis:*
Name dc:type.qualificationname
Ph.D
Level dc:type.qualificationlevel
doctoral
Grantor dc:publisher.institution
University of York
Year dc:date.issued
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Yang, Jingbo
Advisor dc:contributor.advisor
  • Bors, Adrian
ORCID person:identifier.orcid

Rights

Language dc:language.dcterms_iso639_2
eng

Chain of custody

source
Harvested from
White Rose University Consortium
Base URL
etheses.whiterose.ac.uk/cgi/oai2
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
related terms
citation

Yang, Jingbo. Long video generation using the VAE-GAN method. doctoral thesis, University of York, 2024.