In this thesis, an audio-video database for speaker recognition is constructed using a digital camcorder. Motion pictures of fifteen hundred speakers are recorded in three different sessions in the database. For each speaker, 20 still images per session are also derived from the video data. It is hoped that this database can provide an appropriate training and testing mechanism for person identification using both voice and face features.

Speaker or voice recognition is the task of automatically recognizing people from their speech signals. This technique makes it possible to use uttered speech to verify the speaker's identity and control access to secured services. Surveillance, counter-terrorism and homeland security department can collect voice data from telephone conversation without having to access to any other biometric dataset. In this type of scenario it would be beneficial if the confidence level of authentication is high. Other applicable areas include online transactions,database access services, information services, security control for confidential information areas, and remote access to computers. Speaker recognition systems, even though they have been around for four decades, have not been widely considered as standalone systems for biometric security because of their unacceptably low performance, i.e., high false acceptance and true rejection. This thesis focuses on the enhancement of speaker recognition through a combination of intra-modal fusion and accent modeling. Initial enhancement of speaker recognition was achieved through intra-modal hybrid fusion (HF) of likelihood scores generated by Arithmetic Harmonic Sphericity (AHS) and Hidden Markov Model (HMM) techniques. Due to the Contrastive nature of AHS and HMM, we have observed a significant performance improvement of 22% , 6% and 23% true acceptance rate (TAR) at 5% false acceptance rate (FAR), when this fusion technique was evaluated on three different datasets -- YOHO, USF multi-modal biometric and Speech Accent Archive (SAA), respectively. Performance enhancement has been achieved on both the datasets; however performance on YOHO was comparatively higher than that on USF dataset, owing to the fact that USF dataset is a noisy outdoor dataset whereas YOHO is an indoor dataset. In order to further increase the speaker recognition rate at lower FARs, we combined accent information from an accent classification (AC) system with our earlier HF system. Also, in homeland security applications, speaker accent will play a critical role in the evaluation of biometric systems since users will be international in nature. So incorporating accent information into the speaker recognition/verification system is a key component that our study focused on. The proposed system achieved further performance improvements of 17% and 15% TAR at an FAR of 3% when evaluated on SAA and USF multi-modal biometric datasets. The accent incorporation method and the hybrid fusion techniques discussed in this work can also be applied to any other speaker recognition systems.

Ο στόχος ενός συστήματος αυτόματης αναγνώρισης ομιλητή είναι άρρηκτα συνδεδεμένος με την εξαγωγή, το χαρακτηρισμό και την αναγνώριση πληροφοριών σχετικά με την ταυτότητα ενός ομιλητή. Η αναγνώριση ομιλητή αναφέρεται είτε στην ταυτοποίηση είτε στην επιβεβαίωσή του. Συγκεκριμένα, ανάλογα με τη μορφή της απόφασης που επιστρέφει, ένα σύστημα ταυτοποίησης μπορεί να χαρακτηριστεί ως ανοιχτού συνόλου (open-set) ή ως κλειστού συνόλου (closed-set). Αν ένα σύστημα βασιζόμενο σε ένα άγνωστο δείγμα φωνής αποκρίνεται με μια ντετερμινιστικής μορφής απόφαση, εάν το δείγμα ανήκει σε συγκεκριμένο ή σε άγνωστο ομιλητή, το σύστημα χαρακτηρίζεται ως σύστημα ταυτοποίησης ανοιχτού συνόλου. Από την άλλη πλευρά, στην περίπτωση που το σύστημα επιστρέφει τον πιθανότερο ομιλητή, από αυτούς που ήδη είναι καταχωρημένοι στη βάση, από τον οποίο προέρχεται το δείγμα φωνής το σύστημα χαρακτηρίζεται ως σύστημα κλειστού συνόλου. Η ταυτοποίηση συστήματος κλειστού συνόλου, περαιτέρω μπορεί να χαρακτηριστεί ως εξαρτημένη ή ανεξάρτητη από κείμενο, ανάλογα με το εάν το σύστημα γνωρίζει την εκφερόμενη φράση ή εάν αυτό είναι ικανό να αναγνωρίσει τον ομιλητή από οποιαδήποτε φράση που μπορεί αυτός να εκφέρει. Στην εργασία αυτή εξετάζονται και υλοποιούνται αλγόριθμοι αυτόματης αναγνώρισης ομιλητή που βασίζονται σε κλειστού τύπου και ανεξαρτήτως κειμένου συστήματα ταυτοποίησης. Συγκεκριμένα, υλοποιούνται αλγόριθμοι που βασίζονται στην ιδέα της διανυσματικής κβάντισης, τα στοχαστικά μοντέλα και τα νευρωνικά δίκτυα. / The purpose of system of automatic recognition of speaker is unbreakably connected with the export, the characterization and the recognition of information with regard to the identity of speaker. The recognition of speaker is reported or in the identification or in his confirmation. Concretely, depending on the form of decision that returns, a system of identification can be characterized as open-set or as closed-set. If a system based on an unknown sample of voice is replied with deterministic form decision, if the sample belongs in concrete or in unknown speaker, the system is characterized as system of identification of open set. On the other hand, in the case where the system return the more likely speaker than which emanates the sample of voice, the system is characterized as system of closed set. The identification of system of close set, further can be characterized as made dependent or independent from text, depending on whether the system knows the speaking phrase or if this is capable to recognize the speaker from any phrase that can speak. In this work they are examined and they are implemented algorithms of automatic recognition of speaker that are based in closed type and independent text systems of identification. Concretely, are implemented algorithms that are based in the idea of the Vector Quantization, the stochastic models and the neural networks.

Atualmente, diversos trabalhos e aplicações são desenvolvidos com foco na área de reconhecimento computacional de locutores. À medida que o interesse por diversas aplicações reais dentro dessa área emerge, principalmente em biometria, na qual a segurança e a eficácia são de extrema importância, torna-se cada vez mais necessário que estudos sejam feitos, na mesma proporção, visando avaliá-las. Desse modo, a proposta do presente trabalho é a de mensurar a acurácia de um sistema de reconhecimento de locutores baseado em características elementares, isto é, energias de sub-bandas de frequências, em associação com um classificador probabilístico, estudando a viabilidade de extraí-las das transições entre trechos vozeados e não-vozeados (TTVNV) dos sinais. Testes são realizados com diferentes quantidades de locutores e discurso fixado. A acurácia obtida nos testes variam de 20.18% a 92.53%. Os resultados obtidos são comparados e relatados, complementando as afirmações existentes na literatura sobre o uso das TTVNV com dados quantitativos. / Nowadays, many works and applications are developed focusing on computational speaker recognition. As the interest for several real applications within this area emerges, especially in biometrics, where the safety and the efficacy of the applications are extremely important, studies need to be developed in the same proportion, to evaluate the effectiveness of such approaches. Based on that, this work intends to measure the accuracy of a speaker recognition system that uses elementar features, i.e., sub-band frequency energies, associated with a probabilistic classifier, studying the viability of extracting them from the transition between voiced and unvoiced speech tags (TTVNV). Tests are carried out with different numbers of speakers and a text-dependent approach. The accuracy of the tests varies from 20.18% to 92.53%. The results are compared and reported, complementing the existent information on the use of TTVNV with quantitative data.

Le domaine de la reconnaissance automatique du locuteur (RAL) a vu des avancées considérables dans la dernière décennie permettant d’atteindre des taux d’erreurs très faibles dans des conditions contrôlées. Cependant, l’implémentation de cette technologie dans des applications réelles est entravée par la grande dégradation des performances en présence de nuisances acoustiques en phase d’utilisation. Un grand effort a été investi par la communauté de recherche en RAL dans la conception de techniques de compensation des nuisances acoustiques. Ces techniques opèrent à différents niveaux : signal, paramètres acoustiques, modèles ou scores. Avec le développement du paradigme de "variabilité totale", de nouvelles possibilités peuvent être explorées profitant des propriété statistiques simples de l’espace des i-vecteurs. Notre travail de thèse s’inscrit dans ce cadre et propose des techniques de compensation des nuisances acoustiques qui opèrent directement dans le domaine des i-vecteurs. Ces algorithmes utilisent des relations simples entre les i-vecteurs corrompus et leurs versions propres et font abstraction de l’effet réel des nuisances dans cet espace. Afin de mettre en œuvre cette méthodologie, des exemples de données propres / corrompues sont générés artificiellement et utilisés pour construire des algorithmes de compensation des nuisances acoustiques. Ce procédé permet d’éviter les dérivations qui peuvent être complexes, voire très approximatives. Les techniques développées dans cette thèse se divisent en deux classes : La première classe de techniques se base sur un modèle de distorsion dans le domaine des i-vecteurs. Une relation entre la version propre et la version corrompue d’un i-vecteur est posée et un estimateur permettant de transformer un i-vecteur de test corrompu en sa version propre est construit. La deuxième classe de techniques n’utilise aucun modèle de distorsion dans le domaine des i-vecteurs. Elle permet de tenir compte à la fois de la distribution des i-vecteurs propres, corrompus ainsi que la distribution jointe. Des expériences ont été réalisées sur les données bruitées ainsi que les données de courte durée ; donnés de NIST SRE 2008 bruitées/découpées artificiellement ainsi que les données du challenge SITW bruitées naturellement / de courte durée. / Speaker recognition witnessed considerable progress in the last decade, achieving very low error rates in controlled conditions. However, the implementation of this technology in real applications is hampered by the great degradation of performances in presence of acoustic nuisances. A lot of effort has been invested by the research community in the design of nuisance compensation techniques in the past years. These algorithms operate at different levels : signal, acoustic parameters, models or scores. With the development of the "total variability" paradigm, new possibilities can be explored due to the simple statistical properties of the i-vector space. Our work falls within this framework and presents new compensation techniques which operate directly in the i-vector space. These algorithms use simple relationships between corrupted i-vectors and the corresponding clean versions and ignore the real effect of nuisances in this domain. In order to implement this methodology, pairs of clean and corrupted data are artificially generated then used to develop nuisance compensation algorithms. This method avoids making complex derivations and approximations. The techniques developed in this thesis are divided into two classes : The first class of techniques is based on a distortion model in the i-vector space. A relationships between the clean version of an i-vector and its corrupted version is set and an estimator is built to transform a corrupted test i-vector to its clean counterpart. The second class of techniques does not use any distortion model in the i-vectors domain. It takes into account both the distribution of the clean, corrupt i-vectors as well as the joint distribution. Experiments are carried-out on noisy data and short utterances ; artificially corrupted NIST SRE 2008 data and natural SITW (short / noisy segments).

Le domaine de la reconnaissance automatique du locuteur (RAL) recouvre l’ensembledes techniques visant à discriminer des locuteurs à partir de leurs énoncésde voix. Il se classe dans la famille des procédures d’authentification biométrique del’identité. La reconnaissance du locuteur a connu ces dernières années une avancée significativeavec un nouveau concept de représentation de l’énoncé de voix, désignésous le terme de i-vector. Ce type de représentation s’appuie sur le paradigme de modélisationpar mélange de gaussiennes et présente la particularité de se réduire numériquementà un vecteur de dimension faible, au regard des représentations précédentes,et pourtant très discriminant vis à vis du locuteur.Les travaux présentés dans cette thèse s’inscrivent dans ce nouveau contexte. Orientésautour de cette représentation, ils visent à en comprendre et évaluer les hypothèses,les points fondamentaux, le comportement et les limites.Nous avons en premier lieu conduit une analyse statistique sur cette nouvelle représentation.L’étude a porté sur l’effet et l’importance relative des différentes étapes deconstitution et d’exploitation du concept. Cette analyse a permis de mieux comprendreses caractéristiques, mais aussi de faire apparaître des défauts de la représentation quinous ont conduits à mettre en place de nouvelles transformations dans cet espace. L’objectifde ces techniques est de faire converger les données vers des modèles théoriques,à meilleur pouvoir discriminant. Nous recensons et démontrons un certain nombre depropriétés induites par ces transformations, qui justifient leur emploi. En terme de performance,ces techniques réduisent d’un ordre de grandeur de 50% les taux d’erreurdes systèmes basés sur les i-vectors et des postulats gaussiens, permettant notammentd’atteindre par la voie du cadre probabiliste gaussien les meilleurs taux de détectiondans le domaine.Une évaluation générale des composants de la méthode est ensuite détaillée dansce document. Elle met en avant l’importance de certaines étapes, permettant ainsi dedégager, par comparaison à des méthodes alternatives, les approches fondamentalesqui confèrent au concept une valeur de paradigme. Nous montrons la primauté decertaines étapes stratégiques dans la chaîne des traitements, parmi lesquelles les transformationsque nous avons mises en place, et leur relative indépendance aux méthodes et hypothèses adoptées.Des limites de la solution sont mises au jour et exposées dans une étude dite d’anisotropie,qui relativise sa capacité à produire une paramétrisation linéaire globale des variabilitésqui soit optimale.En parallèle de ces investigations, nous avons participé à l’exploration d’un nouveaumodèle alternatif à la solution la plus usuelle de représentation des énoncés devoix. Conçu par J.F. Bonastre, il produit des vecteurs sous forme de clés binaires etfournit les moyens de les comparer, en suivant une voie semi-paramétrique basée surune nouvelle approche de la problématique. Cette exploration a contribué à l’améliorationde ce modèle et à l’ouverture de nouvelles pistes. Elle a été également utile à notreévaluation du concept de i-vector.Les travaux présentés dans ce document contribuent à l’amélioration de ce modèleet à l’ouverture de nouvelles pistes. Ils sont également utiles à notre évaluation duconcept de i-vector.Enfin, quelques aménagements des solutions i-vectors à des cas particuliers ont étémis en place : nous proposons de nouvelles variantes pour gérer la décision sur lesénoncés de courte durée (qui constituent l’un des enjeux actuels du domaine) et sur lesénoncés présentant une divergence a priori (support, durée, langue distincts).L’ensemble de ces travaux vise à mieux circonscrire les pistes de recherche les plusporteuses autour de ce nouveau concept de représentation de la voix humaine / The speaker recognition field covers all the techniques intended to authentify theidentity by using voice utterances. Speaker recognition has experienced in recent yearsa significant step forward with a new concept of representation, referred to as the ivector. This type of representation is based on the Gaussian mixture model paradigmand has the distinguishing feature of being a small size vector compared to previousrepresentations, yet very discriminating towards the speaker.The works presented in this thesis are within that new context. Focused on thisrepresentation, they aim to better understand it and assess its assumptions, highlightits key points, its behaviors and limits.We first carried out a statistical analysis of this new representation. This analysishelped to better understand its characteristics, but also reveal defaults of the representationthat led us to develop new transformations. The goal of these techniques is tomove data towards a theoretical model, having a better accuracy for discrimination.We identify and demonstrate a number of properties of these transformations whichjustify their relevance. In terms of performance, applying these techniques reduce byan order of magnitude of 50% the error rate of systems based on i-vectors and Gaussianassumptions and yield the best detection rate in the field through the Gaussianprobabilistic framework. A complete evaluation of the system components is detailed later in this document.By comparing the fundamental approaches to alternative methods, this evaluationidentifies and highlights the fundamental steps that give the concept a value ofparadigm.We show the primacy of some strategic steps in the process chain, includingour propositions, and their relative independence from methods and assumptions.Limits of the solution are uncovered and exposed in a study of "anisotropy", whichreveals some lack of compliance of i-vector distributions with Gaussian assumptions.Alongside these investigations, we participated in the exploration of a new model,alternative to the most usual statistical representations of utterances, which relies on asemi- parametric representation. Designed by J.F. Bonastre, it produces binary key vectorsand provides the means to compare them. This exploration has contributed to the improvement of this model and opens new gates. It was also helpful to our evaluationof the concept of i -vector.Some adaptations of i-vector approach to special speaker recognition tasks are described: we propose new variants to handle short duration utterances ( which is oneof the current issues in the field ) and to deal with a priori mismatch (for example ofsupport, time or distinct language).We hope that this work will better highlight some of the most promising slopes ofresearch around this new concept of representation for speaker recognition

The objective of this work is to study state-of-the-art deep neural networks based speaker verification systems called x-vectors on various conditions, such as wideband and narrowband data and to develop the system, which is robust to unseen language, specific noise or speech codec. This system takes variable length audio recording and maps it into fixed length embedding which is afterward used to represent the speaker. We compared our systems to BUT's submission to Speakers in the Wild Speaker Recognition Challenge (SITW) from 2016, which used previously popular statistical models - i-vectors. We observed, that when comparing single best systems, with recently published x-vectors we were able to obtain more than 4.38 times lower Equal Error Rate on SITW core-core condition compared to SITW submission from BUT. Moreover, we find that diarization substantially reduces error rate when there are multiple speakers for SITW core-multi condition but we could not see the same trend on NIST SRE 2018 VAST data.

The content of this work is focused on the neural network per speaker recognition. The work deals with problems of processing speech signal and there are introduction some types of neural network. The part of work was made database of records from speakers with have various sex and ages. The train and test group was made from the database. For classifier were suggested afterwards. One of them was nominated on base Gaussian mixture model and three of them were nominated on neural. This system was tested and analyzed on the basis of age, gender and both criterions each other at the end. Attention is focused on choice suitable feature in each mission of classification at the same time. At the end of work are introduced results of analysis for individual groups and features. The most suitable features are diagnosed from given mission of classification and the most prosperous classifier.

My diploma theses deals with the problem of the speaker recognition. The basic theory of this problem is described in the text as well as model and implementation of the system for speaker recognition. The scope of the system is to recognize up to three speakers. The theory is based on calculation parameters for speaker recognition and processing of voice. Program is made in Matlab as a independent application and it has got Czech and English interface.

Tato práce se zabývá extrakcí vhodných příznaků pro rozpoznávání řečníka z delších časových úseků. Po představení současných technik pro extrakci takových příznaků navrhujeme a popisujeme novou metodu pracující v časovém rozsahu fonémů a využívající známou techniku i-vektorů. Velké úsilí bylo vynaloženo na nalezení vhodné reprezentace temporálních příznaků, díky kterým by mohly být systémy pro rozpoznávání řečníka robustnější, zejména modelování prosodie. Náš přístup nemodeluje explicitně žádné specifické temporální parametry řeči, namísto toho používá kookurenci řečových rámců jako zdroj temporálních příznaků. Tuto techniku testujeme a analyzujeme na řečové databázi NIST SRE 2008. Z výsledků bohužel vyplývá, že pro rozpoznávání řečníka tato technika nepřináší očekávané zlepšení. Tento fakt diskutujeme a analyzujeme ke konci práce.