Georgia Institute of Technology
Incorporating Geometric and Consistency Constraints into Deep Models for Robust Phase Reconstruction and Speech Enhancement
Abstract
dc:description.abstractPhase reconstruction of short-term Fourier transform (STFT) spectra remains a central challenge in speech generation tasks. Although early studies emphasized magnitude over phase, recent work has shown that accurate phase reconstruction is crucial for reducing artifacts and distortions that degrade speech quality in enhancement (SE). With the rapid advancement of deep neural networks (DNNs), recent research has aimed to estimate both the magnitude and phase of clean spectrograms simultaneously. However, most approaches directly predict the phase spectrogram—a task made difficult by the phase’s unstructured nature, wrapping ambiguities, and extreme sensitivity to time shifts. Such direct estimation also overlooks alternative phase configurations that can yield perceptually valid speech. This dissertation proposes a new framework for phase estimation that overcomes these limitations and demonstrates its effectiveness across multiple DNN-based SE models. We begin by introducing the first deep state-space-based SE model operating on complex-valued spectrograms. While it surpasses baseline models with a compact U-Net architecture, its estimated phase offers limited improvement over the noisy phase, underscoring the difficulty of direct phase prediction. To address this, we develop a novel explicit consistency-preserving loss that leverages the observation that perceptually high-quality speech arises when magnitude and phase are mutually consistent. Building on this insight, we integrate geometric constraints under additive noise conditions with the consistency principle, resulting in the Multi-Sourced Griffin-Lim Algorithm (MSGLA). MSGLA jointly refines speech and noise phases through iterative updates guided by DNN-estimated magnitudes and geometric relationships, outperforming direct phase estimation and prior geometric methods. Finally, we extend these ideas to a large-scale generative pretraining framework that models the distribution of clean speech spectrograms and incorporates the consistency-based phase loss during training.
Degree
thesis:*- Name thesis:degree_name
- Electrical and Computer Engineering, PhD
- Department thesis:degree_department
- Electrical and Computer Engineering
- Grantor
- Georgia Institute of Technology
- Year dc:date.issued
- 2025
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Ku, Pin-Jui
- Advisor dc:contributor.advisor
-
- Lee, Chin-Hui
- Committee members dc:contributor.committeemember
-
- Heck, Larry
- Moore, Elliot
- Anderson, David
- ORCID person:identifier.orcid
Rights
- Language dc:language.iso
- English
Identifiers
dc:identifier.*- Handle dc:identifier.uri
- https://hdl.handle.net/1853/80222