Back to search

University of Sheffield

Energy-based models for speech synthesis

Abstract

dc:description.abstract

Speech synthesis is an essential technology in a range of widely used applications. Recent progress in speech synthesis has largely been driven by autoregressive (AR) and non-autoregressive (NAR) models that generate highly natural speech but can still produce inconsistencies. This thesis is motivated by the need to address these inconsistencies using generative models, particularly energy-based models (EBMs), and investigates how EBMs can be made practical by connecting them to related frameworks such as diffusion and flow-matching methods. This thesis presents a study on applying energy-based models for speech synthesis. The primary objective of this research is to train energy-based TTS models with commonly used noise contrastive estimation (NCE). During this development the close connection between EBMs and diffusion models developed at the same time became apparent. Building on this connection, score matching, which does not rely on noisy samples, was investigated as an alternative training approach. However, score matching approaches, such as sliced score matching (SSM), is complicated to implement and requires careful tuning. A new loss function is proposed in this thesis to simplify SSM and achieve similar performance. This loss function has close connection to flow matching models which further strengthens the link between EBMs and diffusion like models. The results presented in this thesis show that the proposed methods can effectively improve the quality of synthesized speech compared to pre-trained TTS models.

Degree

thesis:*
Name dc:type.qualificationname
Ph.D
Level dc:type.qualificationlevel
doctoral
Grantor dc:publisher.institution
University of Sheffield
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Sun, Wanli
Advisor dc:contributor.advisor
  • Ragni, Anton

Rights

Language dc:language.dcterms_iso639_2
eng

Chain of custody

source
Harvested from
White Rose University Consortium
Base URL
etheses.whiterose.ac.uk/cgi/oai2
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
related terms
citation

Sun, Wanli. Energy-based models for speech synthesis. doctoral thesis, University of Sheffield, 2026.