The field of speech recognition has during the last decade left the re- search stage and found its way in to the public market. Most computers and mobile phones sold today support dictation and transcription in a number of chosen languages.  Swedish is often not one of them. In this thesis, which is executed on behalf of the Swedish Radio, an Automatic Speech Recognition model for Swedish is trained and the performance evaluated. The model is built using the open source toolkit Kaldi.  Two approaches of training the acoustic part of the model is investigated. Firstly, using Hidden Markov Model and Gaussian Mixture Models and secondly, using Hidden Markov Models and Deep Neural Networks. The later approach using deep neural networks is found to achieve a better performance in terms of Word Error Rate. / De senaste åren har olika tillämpningar inom människa-dator interaktion och främst taligenkänning hittat sig ut på den allmänna marknaden. Många system och tekniska produkter stöder idag tjänsterna att transkribera tal och diktera text. Detta gäller dock främst de större språken och sällan finns samma stöd för mindre språk som exempelvis svenskan. I detta examensprojekt har en modell för taligenkänning på svenska ut- vecklas. Det är genomfört på uppdrag av Sveriges Radio som skulle ha stor nytta av en fungerande taligenkänningsmodell på svenska. Modellen är utvecklad i ramverket Kaldi. Två tillvägagångssätt för den akustiska träningen av modellen är implementerade och prestandan för dessa två är evaluerade och jämförda. Först tränas en modell med användningen av Hidden Markov Models och Gaussian Mixture Models och slutligen en modell där Hidden Markov Models och Deep Neural Networks an- vänds, det visar sig att den senare uppnår ett bättre resultat i form av måttet Word Error Rate.

Dieser Artikel enthält vier Kapitel. Das erste Kapitel ist berechtigt, '''' lokalen Quantil Regression"und seine Zusammenfassung: Quantil Regression ist eine Technik, bedingte Quantil Kurven zu schätzen. Es bietet ein umfassendes Bild über ein Antwort-Kontingent auf erklärenden Variablen. In einem Rahmen flexible Modellierung ist eine besondere Form der bedingten Quantil-Kurve nicht von vornherein festgelegt. Dies motiviert eine lokale parametrische anstatt einer globalen feste Modell passend Ansatz. Eine nichtparametrische Glättung Schätzung der bedingte Quantil Kurve erfordert, zwischen lokalen Krümmung und stochastische auszugleichen Variabilität. In den ersten Essay empfehlen wir eine lokale Modellauswahl Technik, die eine adaptive Schätzung der bedingte bietet Quantil-Regression-Kurve bei jedem Entwurf-Punkt. Theoretische Ergebnisse behaupten, dass das vorgeschlagene adaptive Verfahren als führt gut als Orakel die würde das Risiko der lokalen Abschätzung für die Aufgabenstellung minimieren. Wir veranschaulichen die Leistung der Trolle. / This article includes four chapters. The first chapter is entitled ``Local Quantile Regression", and its summary: Quantile regression is a technique to estimate conditional quantile curves. It provides a comprehensive picture of a response contingent on explanatory variables. In a flexible modeling framework, a specific form of the conditional quantile curve is not a priori fixed. This motivates a local parametric rather than a global fixed model fitting approach. A nonparametric smoothing estimate of the conditional quantile curve requires to balance between local curvature and stochastic variability. In the first essay, we suggest a local model selection technique that provides an adaptive estimate of the conditional quantile regression curve at each design point. Theoretical results claim that the proposed adaptive procedure performs as good as an oracle which would minimize the local estimation risk for the problem at hand. We illustrate the performance of the procedure by an extensive simulation study and consider a couple of applications: to tail dependence analysis for the Hong Kong stock market and to analysis of the distributions of the risk factors of temperature dynamics.

Being one of the fastest growing fields of engineering, wireless has gained the attention of researchers and commercial businesses all over the world. Extensive research is underway to improve the performance of existing systems and to introduce cutting edge wireless technologies that can make high speed wireless communications possible. The first part of this dissertation deals with discrete channel models that are used for simulating error traces produced by wireless channels. Most of the time, wireless channels have memory and we rely on discrete time Markov models to simulate them. The primary advantage of using these models is rapid experimentation and prototyping. Efficient estimation of the parameters of a Markov model (including its number of states) is important to reproducing and/or forecasting channel statistics accurately. Although the parameter estimation of Markov processes has been studied extensively, its order estimation problem has been addressed only recently. In this report, we investigate the existing order estimation techniques for Markov chains and hidden Markov models. Performance comparison with semi-hidden Markov models is also discussed. Error source modeling in slow and fast fading conditions is also considered in great detail. Cognitive Radio is an emerging technology in wireless communications that can improve the utilization of radio spectrum by incorporating some intelligence in its design. It can adapt with the environment and can change its particular transmission or reception parameters to execute its tasks without interfering with the licensed users. One problem that CR network usually faces is the difficulty in detecting and classifying its low power signal that is present in the environment. Most of the time traditional energy detection techniques fail to detect these signals because of their low SNRs. In the second part of this thesis, we address this problem by using higher order statistics of incoming signals and classifying them by using the pattern recognition capabilities of HMMs combined with cased-based learning approach. This dissertation also deals with dynamic spectrum allocation in cognitive radio using HMMs. CR networks that are capable of using frequency bands assigned to licensed users, apart from utilizing unlicensed bands such as UNII radio band or ISM band, are also called Licensed Band Cognitive Radios. In our novel work, the dynamic spectrum management or dynamic frequency allocation is performed by the help of HMM predictions. This work is based on the idea that if Markov models can accurately model spectrum usage patterns of different licensed users, then it should also correctly predict the spectrum holes and use these frequencies for its data transmission. Simulations have shown that HMMs prediction results are quite accurate and can help in avoiding CR interference with the primary licensed users and vice versa. At the same time, this helps in sending its data over these channels more reliably. / Ph. D.

Thesis (PhD (Electrical and Electronic Engineering))--University of Stellenbosch, 2007. / Higher-order Markov models are more powerful than first-order models, but suffer from an exponential increase in model parameters with order, which leads to data scarcity problems during training. A more efficient approach is to use mixed-order Markov models, which model data sequences with contexts of different lengths. This study proposes two algorithms for inferring mixed-order Markov chains and hidden Markov models (HMMs), respectively. The basis of these algorithms is the prediction suffix tree (PST), an efficient representation of a mixed-order Markov chain. The smallest encoded context tree (SECT) algorithm constructs PSTs from data, based on the minimum description length principle. It has no user-specifiable parameters to tune, and will expand the depth of the resulting PST as far as the data set allows it, making it a self-bounded algorithm. It is also faster than the original PST inference algorithm. The hidden SECT algorithm replaces the underlying Markov chain of an HMM with a prediction suffix tree, which is inferred using SECT. The algorithm is efficient and integrates well with standard techniques. The properties of the SECT and hidden SECT algorithms are verified on synthetic data. The hidden SECT algorithm is also compared with a fixed-order HMM training algorithm on an automatic language recognition task, where the resulting mixed-order HMMs are shown to be smaller and train faster than the fixed-order models, for similar classification accuracies.

Proteins are the fundamental unit of computation and signal processing in biological systems. A quantitative understanding of protein biophysics is of paramount importance, since even slight malfunction of proteins can lead to diverse and severe disease states. However, developing accurate and useful mechanistic models of protein function can be strikingly elusive. I demonstrate that the adoption of Bayesian statistical methods can greatly aid in modeling protein systems. I first discuss the pitfall of parameter non-identifiability and how a Bayesian approach to modeling can yield reliable and meaningful models of molecular systems. I then delve into a particular case of non-identifiability within the context of an emerging experimental technique called single molecule photobleaching. I show that the interpretation of this data is non-trivial and provide a rigorous inference model for the analysis of this pervasive experimental tool. Finally, I introduce the use of nonparametric Bayesian inference for the analysis of single molecule time series. These methods aim to circumvent problems of model selection and parameter identifiability and are demonstrated with diverse applications in single molecule biophysics. The adoption of sophisticated inference methods will lead to a more detailed understanding of biophysical systems. / text

Nous présentons une approche de reconnaissance d'écriture manuscrite à partir de champs de Markov cachés et fondée sur une analyse entièrement bidimensionnelle de l'écriture. Son originalité réside dans la combinaison d'une analyse fenêtrée de l'image, d'une modélisation markovienne et dans la mise en oeuvre de la programmation dynamique 2D qui permet un décodage rapide et optimal des champs de Markov. Un aspect important de ces travaux est la méthodologie de développement employée qui est centrée sur l'évaluation systématique des apports algorithmiques et des paramètres utilisés. Ces algorithmes sont en partie empruntés aux techniques utilisées dans le domaine de la reconnaissance de la parole et sont très génériques.<br /><br />L'approche proposée est validée sur deux applications correspondant à des bases de données standard et librement disponibles. L'application de cette méthode extrêmement générique à une tâche de reconnaissance de chiffres manuscrits a permis d'obtenir des résultats comparables à ceux de l'état de l'art. L'application à une tâche de reconnaissance de mots manuscrits a permis de confirmer que l'extension de cette approche à des tâches plus complexes était naturelle.<br /><br />L'ensemble de cette recherche a démontré la validité de l'approche développée qui apparaît comme candidate au statut d'approche standard pour plusieurs problèmes de vision. En outre, elle ouvre la voie à de très nombreux développements concernant la tâche de traitement de l'écriture manuscrite et des améliorations significatives pourraient encore être apportées en recourant à d'autres principes issus du traitement de la parole et du langage. D'autres tâches comme la segmentation d'image devraient tirer avantage de la robustesse et de la faculté d'apprentissage de la modélisation que nous proposons.

The main objective of this thesis was the synthesis of speech synchronised motion, in particular head motion. The hypothesis that head motion can be estimated from the speech signal was confirmed. In order to achieve satisfactory results, a motion capture data base was recorded, a definition of head motion in terms of articulation was discovered, a continuous stream mapping procedure was developed, and finally the synthesis was evaluated. Based on previous research into non-verbal behaviour basic types of head motion were invented that could function as modelling units. The stream mapping method investigated in this thesis is based on Hidden Markov Models (HMMs), which employ modelling units to map between continuous signals. The objective evaluation of the modelling parameters confirmed that head motion types could be predicted from the speech signal with an accuracy above chance, close to 70%. Furthermore, a special type ofHMMcalled trajectoryHMMwas used because it enables synthesis of continuous output. However head motion is a stochastic process therefore the trajectory HMM was further extended to allow for non-deterministic output. Finally the resulting head motion synthesis was perceptually evaluated. The effects of the “uncanny valley” were also considered in the evaluation, confirming that rendering quality has an influence on our judgement of movement of virtual characters. In conclusion a general method for synthesising speech-synchronised behaviour was invented that can applied to a whole range of behaviours.

The majority of automatic speech recognition (ASR) systems rely on hidden Markov models (HMM), in which the output distribution associated with each state is modelled by a mixture of diagonal covariance Gaussians. Dynamic information is typically included by appending time-derivatives to feature vectors. This approach, whilst successful, makes the false assumption of framewise independence of the augmented feature vectors and ignores the spatial correlations in the parametrised speech signal. This dissertation seeks to address these shortcomings by exploring acoustic modelling for ASR with an application of a form of state-space model, the linear dynamic model (LDM). Rather than modelling individual frames of data, LDMs characterize entire segments of speech. An auto-regressive state evolution through a continuous space gives a Markovian model of the underlying dynamics, and spatial correlations between feature dimensions are absorbed into the structure of the observation process. LDMs have been applied to speech recognition before, however a smoothed Gauss-Markov form was used which ignored the potential for subspace modelling. The continuous dynamical state means that information is passed along the length of each segment. Furthermore, if the state is allowed to be continuous across segment boundaries, long range dependencies are built into the system and the assumption of independence of successive segments is loosened. The state provides an explicit model of temporal correlation which sets this approach apart from frame-based and some segment-based models where the ordering of the data is unimportant. The benefits of such a model are examined both within and between segments. LDMs are well suited to modelling smoothly varying, continuous, yet noisy trajectories such as found in measured articulatory data. Using speaker-dependent data from the MOCHA corpus, the performance of systems which model acoustic, articulatory, and combined acoustic-articulatory features are compared. As well as measured articulatory parameters, experiments use the output of neural networks trained to perform an articulatory inversion mapping. The speaker-independent TIMIT corpus provides the basis for larger scale acoustic-only experiments. Classification tasks provide an ideal means to compare modelling choices without the confounding influence of recognition search errors, and are used to explore issues such as choice of state dimension, front-end acoustic parametrization and parameter initialization. Recognition for segment models is typically more computationally expensive than for frame-based models. Unlike frame-level models, it is not always possible to share likelihood calculations for observation sequences which occur within hypothesized segments that have different start and end times. Furthermore, the Viterbi criterion is not necessarily applicable at the frame level. This work introduces a novel approach to decoding for segment models in the form of a stack decoder with A* search. Such a scheme allows flexibility in the choice of acoustic and language models since the Viterbi criterion is not integral to the search, and hypothesis generation is independent of the particular language model. Furthermore, the time-asynchronous ordering of the search means that only likely paths are extended, and so a minimum number of models are evaluated. The decoder is used to give full recognition results for feature-sets derived from the MOCHA and TIMIT corpora. Conventional train/test divisions and choice of language model are used so that results can be directly compared to those in other studies. The decoder is also used to implement Viterbi training, in which model parameters are alternately updated and then used to re-align the training data.

Explorations into the organisation of spontaneous activity within the brain have demonstrated the existence of networks of temporally correlated activity, consisting of brain areas that share similar cognitive or sensory functions. These so-called resting state networks (RSNs) emerge spontaneously during rest and disappear in response to overt stimuli or cognitive demands. In recent years, the study of RSNs has emerged as a valuable tool for probing brain function, both in the healthy brain and in disorders such as schizophrenia, Alzheimer’s disease and Parkinson’s disease. However, analyses of these networks have so far been limited, in part due to assumptions that the patterns of neuronal activity that underlie these networks remain constant over time. Moreover, the majority of RSN studies have used functional magnetic resonance imaging (fMRI), in which slow fluctuations in the level of oxygen in the blood are used as a proxy for the activity within a given brain region. In this thesis we develop the use of magnetoencephalography (MEG) to study resting state functional connectivity. Unlike fMRI, MEG provides a direct measure of neuronal activity and can provide novel insights into the temporal dynamics that underlie resting state activity. In particular, we focus on the application of non- stationary analysis methods, which are able to capture fast temporal changes in activity. We first develop a framework for preprocessing MEG data and measuring interactions within different RSNs (Chapter 3). We then extend this framework to assess temporal variability in resting state functional connectivity by applying time- varying measures of interactions and show that within-network functional connectivity is underpinned by non-stationary temporal dynamics (Chapter 4). Finally we develop a data driven approach based on a hidden Markov model for inferring short lived connectivity states from resting state and task data (Chapter 5). By applying this approach to data from multiple subjects we reveal transient states that capture short lived patterns of neuronal activity (Chapter 6).

At its core, a nanopore detector has a nanometer-scale biological membrane across which a voltage is applied. The voltage draws a DNA molecule into an á-hemolysin channel in the membrane. Consequently, a distinctive channel current blockade signal is created as the molecule flexes and interacts with the channel. This flexing of the molecule is characterized by different blockade levels in the channel current signal. Previous experiments have shown that a nanopore detector is sufficiently sensitive such that nearly identical DNA molecules were classified successfully using machine learning techniques such as Hidden Markov Models and Support Vector Machines in a channel current based signal analysis platform [4-9]. In this paper, methods for improving feature extraction are presented to improve both classification and to provide biologists and chemists with a better understanding of the physical properties of a given molecule.