Apesar da ampla diversidade de línguas indígenas no Brasil, poucas pesquisas estudam estas línguas e suas relações. Inúmeros esforços têm sido dedicados a procurar similaridades entre as palavras das línguas indígenas e classificá-las em famílias de línguas. Seguindo a classificação mais aceita das línguas indígenas do Brasil, esta pesquisa propõe comparar palavras de 10 línguas indígenas brasileiras. Para isso, considera-se que estas palavras são sinais de fala e estima-se a função de distribuição de probabilidade (PDF) de cada palavra, usando um modelo de mistura de gaussianas (GMM). A PDF foi considerada um modelo para representar as palavras. Os modelos foram comparados utilizando medidas de distância para construir estruturas hierárquicas que evidenciaram possíveis relações entre as palavras. Seguindo esta linha, a hipótese levantada nesta pesquisa é que as PDFs baseadas em GMM conseguem caracterizar as palavras das línguas indígenas, permitindo o emprego de medidas de distância entre elas para estabelecer relações entre as palavras, de forma que tais relações confirmem algumas das classificações. Os parâmetros do GMM foram calculados utilizando o algoritmo Maximização da Expectância (em inglês, Expectation Maximization (EM)). A divergência Kullback Leibler (KL) foi empregada para medir semelhança entre as PDFs. Esta divergência serve de base para estabelecer as estruturas hierárquicas que ilustram as relações entre os modelos. A estimativa da PDF, baseada em GMM foi testada com o auxílio de sinais simulados, sendo possível confirmar que os parâmetros obtidos são próximos dos originais. Foram implementadas várias medidas de distância para avaliar se a semelhança entre os modelos estavam determinadas pelos modelos e não pelas medidas adotadas neste estudo. Os resultados de todas as medidas foram similares, somente foi observada alguma diferença nos agrupamentos realizados pela distância C2, por isso foi proposta como complemento da divergência KL. Estes resultados sugerem que as relações entre os modelos dependem das suas características, não das métricas de distância selecionadas no estudo e que as PDFs baseadas em GMM, conseguem fazer uma caracterização adequada das palavras. Em geral, foram observados agrupamentos entre palavras que pertenciam a línguas de um mesmo tronco linguístico, assim como se observou uma tendência a incluir línguas isoladas nos agrupamentos dos troncos linguísticos. Palavras que pertenciam a determinada língua apresentaram um comportamento padrão, sendo identificadas por esse tipo de comportamento. Embora os resultados para as palavras das línguas indígenas sejam inconclusivos, considera-se que o estudo foi útil para aumentar o conhecimento destas 10 línguas estudadas, propondo novas linhas de pesquisas dedicadas à análise destas palavras. / Although there exists a large diversity of indigenous languages in Brazil, there are few researches on these languages and their relationships. Numerous efforts have been dedicated to search for similarities among words of indigenous languages to classify them into families. Following the most accepted classification of Brazilian indigenous languages, this research proposes to compare words of 10 Brazilian indigenous languages. The words of the indigenous languages are considered speech signals and the Probability Distribution Function (PDF) of each word was estimated using the Gaussian Mixture Models (GMM). This estimation was considered a model to represent each word. The models were compared using distance measures to construct hierarchical structures that illustrate possible relationships among words. The hypothesis in this research is that the estimation of the PDF, based on GMM can characterize the words of indigenous languages, allowing the use of distance measures between the PDFs to establish relationships among the words and confirm some of the classifications. The Expectation Maximization algorithm (EM) was implemented to estimate the parameters that describe the GMM. The Kullback Leibler (KL) divergence was used to measure similarities between two PDFs. This divergence is the basis to establish the hierarchical structures that show the relationships among the models. The PDF estimation, based on GMM was tested using simulated signals, allowing confirming the useful approximation of the original parameters. Several distance measures were implemented to prove that the similarities among the models depended on the model of each word, and not on the distance measure adopted in this study. The results of all measures were similar, however, as the clustering results of the C2 distances showed some differences from the other clusters, C2 distance was proposed to complement the KL divergence. The results suggest that the relationships between models depend on their characteristics, and not on the distance measures selected in this study, and the PDFs based on GMM can properly characterize the words. In general, relations among languages that belong to the same linguistic branch were illustrated, showing a tendency to include isolated languages in groups of languages that belong to the same linguistic branches. As the GMM of some language families presents a standard behavior, it allows identifying each family. Although the results of the words of indigenous languages are inconclusive, this study is considered very useful to increase the knowledge of these types of languages and to propose new research lines directed to analyze this type of signals.

La détection d'objets en mouvement représente une étape très importante pour de nombreuses applications telles que l'analyse du comportement humain pour la surveillance visuelle, la reconnaissance d'action par modèle, le suivi du trafic routier, etc. La soustraction d'arrière-plan est une approche populaire, mais difficile étant donnée qu'elle doit surmonter de nombreux obstacles, comme l'évolution dynamique du fond, les variations de luminosité, les occlusions, etc. Dans les travaux présentés, nous nous sommes intéressés à ce problème de segmentation objets/fond, avec une modélisation floue de type-2 pour gérer l'imprécision du modèle et des données. La méthode proposée modélise l'état de chaque pixel à l'aide d'un modèle de mélange de gaussiennes imprécis et évolutif, qui est exploité par plusieurs classifieurs flous pour finalement estimer la classe du pixel à chaque image. Plus précisément, cette décision prend en compte l'historique de son évolution, mais aussi son voisinage spatial et ses éventuels déplacements dans les images précédentes. Puis nous avons comparé la méthode proposée avec d'autres méthodes proches, notamment des méthodes basées sur un modèle de mélanges gaussiens, des méthodes basées floues, ou de type ACP. Cette comparaison nous a permis de situer notre méthode par rapport à l'existant et de proposer quelques perspectives à ce travail. / Moving objects detection is a very important step for many applications such as human behavior analysis surveillance, model-based action recognition, road traffic monitoring, etc. Background subtraction is a popular approach, but difficult given that it must overcome many obstacles, such as dynamic background changes, brightness variations, occlusions, and so on. In the presented works, we focused on this problem of objects/background segmentation, using a type-2 fuzzy modeling to manage the inaccuracy of the model and the data. The proposed method models the state of each pixel using an imprecise and scalable Gaussian mixture model, which is exploited by several fuzzy classifiers to ultimately estimate the pixel class at each image. More precisely, this decision takes into account the history of its evolution, but also its spatial neighborhood and its possible displacements in the preceding images. Then we compared the proposed method with other close methods, including methods based on a gaussian mixture model, fuzzy based methods, or ACP type methods. This comparison allowed us to assess its good performances, and to propose some perspectives to this work.

Acoustic analysis of animal vocalizations has been widely used to identify the presence of individual species, classify vocalizations, identify individuals, and determine gender. In this work automatic identification of speaker and gender of mice from ultrasonic vocalizations and speaker identification of meerkats from their Close calls is investigated. Feature extraction was implemented using Greenwood Function Cepstral Coefficients (GFCC), designed exclusively for extracting features from animal vocalizations. Mice ultrasonic vocalizations were analyzed using Gaussian Mixture Models (GMM) which yielded an accuracy of 78.3% for speaker identification and 93.2% for gender identification. Meerkat speaker identification with Close calls was implemented using Gaussian Mixture Models (GMM) and Hidden Markov Models (HMM), with an accuracy of 90.8% and 94.4% respectively. The results obtained shows these methods indicate the presence of gender and identity information in vocalizations and support the possibility of robust gender identification and individual identification using bioacoustic data sets.

In this master's thesis, hidden Markov models (HMM) are evaluated as a tool for forecasting movements in a currency cross. With an ever increasing electronic market, making way for more automated trading, or so called algorithmic trading, there is constantly a need for new trading strategies trying to find alpha, the excess return, in the market. HMMs are based on the well-known theories of Markov chains, but where the states are assumed hidden, governing some observable output. HMMs have mainly been used for speech recognition and communication systems, but have lately also been utilized on financial time series with encouraging results. Both discrete and continuous versions of the model will be tested, as well as single- and multivariate input data. In addition to the basic framework, two extensions are implemented in the belief that they will further improve the prediction capabilities of the HMM. The first is a Gaussian mixture model (GMM), where one for each state assign a set of single Gaussians that are weighted together to replicate the density function of the stochastic process. This opens up for modeling non-normal distributions, which is often assumed for foreign exchange data. The second is an exponentially weighted expectation maximization (EWEM) algorithm, which takes time attenuation in consideration when re-estimating the parameters of the model. This allows for keeping old trends in mind while more recent patterns at the same time are given more attention. Empirical results shows that the HMM using continuous emission probabilities can, for some model settings, generate acceptable returns with Sharpe ratios well over one, whilst the discrete in general performs poorly. The GMM therefore seems to be an highly needed complement to the HMM for functionality. The EWEM however does not improve results as one might have expected. Our general impression is that the predictor using HMMs that we have developed and tested is too unstable to be taken in as a trading tool on foreign exchange data, with too many factors influencing the results. More research and development is called for.

Clock synchronization between any two nodes in a Wireless Sensor Network (WSNs) is generally accomplished through exchanging messages and adjusting clock offset and skew parameters of each node’s clock. To cope with unknown network message delays, the clock offset and skew estimation schemes have to be reliable and robust in order to attain long-term synchronization and save energy. A joint clock offset and skew estimation scheme is studied and developed based on the Gaussian Mixture Kalman Particle Filter (GMKPF). The proposed estimation scheme is shown to be a more flexible alternative than the Gaussian Maximum Likelihood Estimator (GMLE) and the Exponential Maximum Likelihood Estimator (EMLE), and to be a robust estimation scheme in the presence of non-Gaussian/nonexponential random delays. This study also includes a sub optimal method called Maximum Likelihood-like Estimator (MLLE) for Gaussian and exponential delays. The computer simulations illustrate that the scheme based on GMKPF yields better results in terms of Mean Square Error (MSE) relative to GMLE, EMLE, GMLLE, and EMLLE, when the network delays are modeled as non-Gaussian/non-exponential distributions or as a mixture of several distributions.

<p>In this master's thesis, hidden Markov models (HMM) are evaluated as a tool for forecasting movements in a currency cross. With an ever increasing electronic market, making way for more automated trading, or so called algorithmic trading, there is constantly a need for new trading strategies trying to find alpha, the excess return, in the market.</p><p>HMMs are based on the well-known theories of Markov chains, but where the states are assumed hidden, governing some observable output. HMMs have mainly been used for speech recognition and communication systems, but have lately also been utilized on financial time series with encouraging results. Both discrete and continuous versions of the model will be tested, as well as single- and multivariate input data.</p><p>In addition to the basic framework, two extensions are implemented in the belief that they will further improve the prediction capabilities of the HMM. The first is a Gaussian mixture model (GMM), where one for each state assign a set of single Gaussians that are weighted together to replicate the density function of the stochastic process. This opens up for modeling non-normal distributions, which is often assumed for foreign exchange data. The second is an exponentially weighted expectation maximization (EWEM) algorithm, which takes time attenuation in consideration when re-estimating the parameters of the model. This allows for keeping old trends in mind while more recent patterns at the same time are given more attention.</p><p>Empirical results shows that the HMM using continuous emission probabilities can, for some model settings, generate acceptable returns with Sharpe ratios well over one, whilst the discrete in general performs poorly. The GMM therefore seems to be an highly needed complement to the HMM for functionality. The EWEM however does not improve results as one might have expected. Our general impression is that the predictor using HMMs that we have developed and tested is too unstable to be taken in as a trading tool on foreign exchange data, with too many factors influencing the results. More research and development is called for.</p>

Modern speech communication technologies expose users to perceptual quality degradations that were not experienced earlier with conventional telephone systems. Since perceived speech quality is a major contributor to the end user's perception of quality of service, speech quality estimation has become an important research field. In this dissertation, perceptual quality estimators are proposed for several emerging speech communication applications, in particular for i) wireless communications with noise suppression capabilities, ii) wireless-VoIP communications, iii) far-field hands-free speech communications, and iv) text-to-speech systems. First, a general-purpose speech quality estimator is proposed based on statistical models of normative speech behaviour and on innovative techniques to detect multiple signal distortions. The estimators do not depend on a clean reference signal hence are termed ``blind." Quality meters are then distributed along the network chain to allow for both quality degradations and quality enhancements to be handled. In order to improve estimation performance for wireless communications, statistical models of noise-suppressed speech are also incorporated. Next, a hybrid signal-and-link-parametric quality estimation paradigm is proposed for emerging wireless-VoIP communications. The algorithm uses VoIP connection parameters to estimate a base quality representative of the packet switching network. Signal-based distortions are then detected and quantified in order to adjust the base quality accordingly. The proposed hybrid methodology is shown to overcome the limitations of existing pure signal-based and pure link parametric algorithms. Temporal dynamics information is then investigated for quality diagnosis for hands-free speech communications. A spectro-temporal signal representation, where speech and reverberation tail components are shown to be separable, is used for blind characterization of room acoustics. In particular, estimators of reverberation time, direct-to-reverberation energy ratio, and reverberant speech quality are developed. Lastly, perceptual quality estimation for text-to-speech systems is addressed. Text- and speaker-independent hidden Markov models, trained on naturally produced speech, are used to capture normative spectral-temporal information. Deviations from the models, computed by means of a log-likelihood measure, are shown to be reliable indicators of multiple quality attributes including naturalness, fluency, and intelligibility. / Thesis (Ph.D, Electrical & Computer Engineering) -- Queen's University, 2008-12-22 14:54:49.28

O recente campo de Deep Learning introduziu a área de Aprendizagem de Máquina novos métodos baseados em representações distribuídas e abstratas dos dados de treinamento ao longo de estruturas hierárquicas. A organização hierárquica de camadas permite que esses métodos guardem informações distribuídas sobre os sinais sensoriais e criem conceitos com diferentes níveis de abstração para representar os dados de entrada. Este trabalho investiga o impacto de uma estrutura hierárquica inspirada pelas ideias apresentadas em Deep Learning, e com base na Incremental Gaussian Mixture Network (IGMN), uma rede neural probabilística com aprendizagem online e incremental, especialmente adequada para as tarefas de robótica. Como resultado, foi desenvolvida uma arquitetura hierárquica, denominada Hierarchical Incremental Gaussian Mixture Network (HIGMN), que combina dois níveis de IGMNs. As camadas de primeiro nível da HIGMN são capazes de aprender conceitos a partir de dados de diferentes domínios que são então relacionados na camada de segundo nível. O modelo proposto foi comparado com a IGMN em tarefas de robótica, em especial, na tarefa de aprender e reproduzir um comportamento de seguir paredes, com base em uma abordagem de Aprendizado por Demonstração. Os experimentos mostraram como a HIGMN pode executar três diferentes tarefas em paralelo (aprendizagem de conceitos, segmentação de comportamento, e aprendizagem e reprodução de comportamentos) e sua capacidade de aprender um comportamento de seguir paredes e reproduzi-lo em ambientes desconhecidos com novas informações sensoriais. A HIGMN conseguiu reproduzir o comportamento de seguir paredes depois de uma única, simples e curta demonstração do comportamento. Além disso, ela adquiriu conhecimento de diferentes tipos: informações sobre o ambiente, a cinemática do robô, e o comportamento alvo. / The recent field of Deep Learning has introduced to Machine Learning new meth- ods based on distributed abstract representations of the training data throughout hierarchical structures. The hierarchical organization of layers allows these meth- ods to store distributed information on sensory signals and to create concepts with different abstraction levels to represent the input data. This work investigates the impact of a hierarchical structure inspired by ideas on Deep Learning and based on the Incremental Gaussian Mixture Network (IGMN), a probabilistic neural network with an on-line and incremental learning, specially suitable for robotic tasks. As a result, a hierarchical architecture, called Hierarchical Incremental Gaussian Mixture Network (HIGMN), was developed, which combines two levels of IGMNs. The HIGMN first-level layers are able to learn concepts from data of different domains that are then related in the second-level layer. The proposed model was compared with the IGMN regarding robotic tasks, in special, the task of learning and repro- ducing a wall-following behavior, based on a Learning from Demonstration (LfD) approach. The experiments showed how the HIGMN can perform parallely three different tasks concept learning, behavior segmentation, and learning and repro- ducing behaviors and its ability to learn a wall-following behavior and to perform it in unknown environments with new sensory information. HIGMN could reproduce the wall-following behavior after a single, simple, and short demonstration of the behavior. Moreover, it acquired different types of knowledge: information on the environment, the robot kinematics, and the target behavior.

O recente campo de Deep Learning introduziu a área de Aprendizagem de Máquina novos métodos baseados em representações distribuídas e abstratas dos dados de treinamento ao longo de estruturas hierárquicas. A organização hierárquica de camadas permite que esses métodos guardem informações distribuídas sobre os sinais sensoriais e criem conceitos com diferentes níveis de abstração para representar os dados de entrada. Este trabalho investiga o impacto de uma estrutura hierárquica inspirada pelas ideias apresentadas em Deep Learning, e com base na Incremental Gaussian Mixture Network (IGMN), uma rede neural probabilística com aprendizagem online e incremental, especialmente adequada para as tarefas de robótica. Como resultado, foi desenvolvida uma arquitetura hierárquica, denominada Hierarchical Incremental Gaussian Mixture Network (HIGMN), que combina dois níveis de IGMNs. As camadas de primeiro nível da HIGMN são capazes de aprender conceitos a partir de dados de diferentes domínios que são então relacionados na camada de segundo nível. O modelo proposto foi comparado com a IGMN em tarefas de robótica, em especial, na tarefa de aprender e reproduzir um comportamento de seguir paredes, com base em uma abordagem de Aprendizado por Demonstração. Os experimentos mostraram como a HIGMN pode executar três diferentes tarefas em paralelo (aprendizagem de conceitos, segmentação de comportamento, e aprendizagem e reprodução de comportamentos) e sua capacidade de aprender um comportamento de seguir paredes e reproduzi-lo em ambientes desconhecidos com novas informações sensoriais. A HIGMN conseguiu reproduzir o comportamento de seguir paredes depois de uma única, simples e curta demonstração do comportamento. Além disso, ela adquiriu conhecimento de diferentes tipos: informações sobre o ambiente, a cinemática do robô, e o comportamento alvo. / The recent field of Deep Learning has introduced to Machine Learning new meth- ods based on distributed abstract representations of the training data throughout hierarchical structures. The hierarchical organization of layers allows these meth- ods to store distributed information on sensory signals and to create concepts with different abstraction levels to represent the input data. This work investigates the impact of a hierarchical structure inspired by ideas on Deep Learning and based on the Incremental Gaussian Mixture Network (IGMN), a probabilistic neural network with an on-line and incremental learning, specially suitable for robotic tasks. As a result, a hierarchical architecture, called Hierarchical Incremental Gaussian Mixture Network (HIGMN), was developed, which combines two levels of IGMNs. The HIGMN first-level layers are able to learn concepts from data of different domains that are then related in the second-level layer. The proposed model was compared with the IGMN regarding robotic tasks, in special, the task of learning and repro- ducing a wall-following behavior, based on a Learning from Demonstration (LfD) approach. The experiments showed how the HIGMN can perform parallely three different tasks concept learning, behavior segmentation, and learning and repro- ducing behaviors and its ability to learn a wall-following behavior and to perform it in unknown environments with new sensory information. HIGMN could reproduce the wall-following behavior after a single, simple, and short demonstration of the behavior. Moreover, it acquired different types of knowledge: information on the environment, the robot kinematics, and the target behavior.

Atualmente diversos sistemas sensoriais fornecem dados em fluxos e essas observações medidas são frequentemente de alta dimensionalidade, ou seja, o número de variáveis medidas é grande, e as observações chegam em sequência. Este é, em particular, o caso de sistemas de visão em robôs. Aprendizagem supervisionada e não-supervisionada com esses fluxos de dados é um desafio, porque o algoritmo deve ser capaz de aprender com cada observação e depois descartá-la antes de considerar a próxima, mas diversos métodos requerem todo o conjunto de dados a fim de estimar seus parâmetros e, portanto, não são adequados para aprendizagem em tempo real. Além disso, muitas abordagens sofrem com a denominada maldição da dimensionalidade (BELLMAN, 1961) e não conseguem lidar com dados de entrada de alta dimensionalidade. Para superar os problemas descritos anteriormente, este trabalho propõe um novo modelo de rede neural probabilístico e incremental, denominado Local Projection Incremental Gaussian Mixture Network (LP-IGMN), que é capaz de realizar aprendizagem perpétua com dados de alta dimensionalidade, ou seja, ele pode aprender continuamente considerando a estabilidade dos parâmetros do modelo atual e automaticamente ajustar sua topologia levando em conta a fronteira do subespaço encontrado por cada neurônio oculto. O método proposto pode encontrar o subespaço intrísico onde os dados se localizam, o qual é denominado de subespaço principal. Ortogonal ao subespaço principal, existem as dimensões que são ruidosas ou que carregam pouca informação, ou seja, com pouca variância, e elas são descritas por um único parâmetro estimado. Portanto, LP-IGMN é robusta a diferentes fontes de dados e pode lidar com grande número de variáveis ruidosas e/ou irrelevantes nos dados medidos. Para avaliar a LP-IGMN nós realizamos diversos experimentos usando conjunto de dados simulados e reais. Demonstramos ainda diversas aplicações do nosso método em tarefas de reconhecimento de imagens. Os resultados mostraram que o desempenho da LP-IGMN é competitivo, e geralmente superior, com outras abordagens do estado da arte, e que ela pode ser utilizada com sucesso em aplicações que requerem aprendizagem perpétua em espaços de alta dimensionalidade. / Nowadays several sensory systems provide data in ows and these measured observations are frequently high-dimensional, i.e., the number of measured variables is large, and the observations are arriving in a sequence. This is in particular the case of robot vision systems. Unsupervised and supervised learning with such data streams is challenging, because the algorithm should be capable of learning from each observation and then discard it before considering the next one, but several methods require the whole dataset in order to estimate their parameters and, therefore, are not suitable for online learning. Furthermore, many approaches su er with the so called curse of dimensionality (BELLMAN, 1961) and can not handle high-dimensional input data. To overcome the problems described above, this work proposes a new probabilistic and incremental neural network model, called Local Projection Incremental Gaussian Mixture Network (LP-IGMN), which is capable to perform life-long learning with high-dimensional data, i.e., it can continuously learn considering the stability of the current model's parameters and automatically adjust its topology taking into account the subspace's boundary found by each hidden neuron. The proposed method can nd the intrinsic subspace where the data lie, which is called the principal subspace. Orthogonal to the principal subspace, there are the dimensions that are noisy or carry little information, i.e., with small variance, and they are described by a single estimated parameter. Therefore, LP-IGMN is robust to di erent sources of data and can deal with large number of noise and/or irrelevant variables in the measured data. To evaluate LP-IGMN we conducted several experiments using simulated and real datasets. We also demonstrated several applications of our method in image recognition tasks. The results have shown that the LP-IGMN performance is competitive, and usually superior, with other stateof- the-art approaches, and it can be successfully used in applications that require life-long learning in high-dimensional spaces.