National Technical University of Athens (NTUA)
Μη γραμμική υπολογιστική μοντελοποίηση φωνής με στοιχεία αεροδυναμικής του φωνητικού σωλήνα
Abstract
dc:descriptionConventional computational speech models usually avoid detailed aerodynamic modeling and determine sound sources in the vocal tract in a phenomenological manner. In this dissertation, a model is developed that allows the computational simulation of important aerodynamic properties that could affect the produced sound. The model exploits recent theoretical and experimental results concerning the flow field in the vocal tract and combines conclusions related to aerodynamics and aeroacoustics of the glottis and the supralaryngeal parts. The aerodynamic-aeroacoustic model is combined with an improved vocal tract acoustics simulation module to achieve articulatory synthesis. To control the articulatory synthesizer, an inversion system was developed that can identify the hidden vocal tract properties given an observed speech signal. The speech inversion system treats speech as essentially an audiovisual process and approximates the complex mapping between the observed information and the vocal tract by means of a switching model. Each submodel is trained via maximum likelihood and canonical correlation analysis. Switching between the submodels is determined by a discrete Markov process. Various alternative audiovisual fusion schemes were investigated that allow interaction between acoustic and optical information at various levels of synchronization. The goal is to recover the underlying vocal tract geometry. Experimental results demonstrate that exploitation of multimodal information by the proposed model clearly benefits inversion results, compared to approaches that exclusively use the one or the other modality. Based on the proposed computational speech model and various articulatory data, mimicing the human speech system becomes possible. More specifically, the articulatory state sequence is modeled as a Markov process and its properties are identified via audiovisual speech inversion. At each articulatory state, given the corresponding description of the vocal tract, we can resynthesize speech via articulatory synthesis. Vocal tract geometry is described by an articulatory model that is trained on X-ray vocal tract data and is properly fitted to the visible part of the tongue in ultrasound data of the mouth cavity. The proposed framework allows broader application and evaluation of the speech acoustics and aerodynamics simulation system and the speech inversion process.
Degree
thesis:*- Grantor dc:publisher
- National Technical University of Athens (NTUA)
- Year dc:date
- 2009
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Katsamanis, Athanasios
- Κατσαμάνης, Αθανάσιος
Subjects
dc:subject × 20- Μοντελοποίηση φωνής
- Αεροδυναμική
- Αεροακουστική
- Σύνθεση φωνής με αρθρωτές
- Επεξεργασία φωνής
- Οπτικο-ακουστική αντιστροφή φωνής
- Μίμηση φωνής
- Επεξεργασία πολυτροπικών δεδομένων άρθρωσης
- Computational speech modeling
- Audiovisual speech inversion
- Articulatory speech synthesis
- Vocal tract aerodynamics
- Vocal tract aeroacoustics
- Speech processing
- Speech mimicing
- Multimodal articulatory data processing
- Επιστήμες Μηχανικού και Τεχνολογία
- Επιστήμη Ηλεκτρολόγου Μηχανικού, Ηλεκτρονικού Μηχανικού, Μηχανικού Η/Υ
- Engineering and Technology
- Electrical Engineering, Electronic Engineering, Information Engineering
Rights
- Language dc:language
- gre
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/18288