This thesis presents a novel lip-reading approach to classifying utterances from video data, without evaluating voice signals. This work addresses two important issues which are • the efficient representation of mouth movement for visual speech recognition • the temporal segmentation of utterances from video. The first part of the thesis describes a robust movement-based technique used to identify mouth movement patterns while uttering phonemes. This method temporally integrates the video data of each phoneme into a 2-D grayscale image named as a motion template (MT). This is a view-based approach that implicitly encodes the temporal component of an image sequence into a scalar-valued MT. The data size was reduced by extracting image descriptors such as Zernike moments (ZM) and discrete cosine transform (DCT) coefficients from MT. Support vector machine (SVM) and hidden Markov model (HMM) were used to classify the feature descriptors. A video speech corpus of 2800 utterances was collected for evaluating the efficacy of MT for lip-reading. The experimental results demonstrate the promising performance of MT in mouth movement representation. The advantages and limitations of MT for visual speech recognition were identified and validated through experiments. A comparison between ZM and DCT features indicates that th e accuracy of classification for both methods is very comparable when there is no relative motion between the camera and the mouth. Nevertheless, ZM is resilient to rotation of the camera and continues to give good results despite rotation but DCT is sensitive to rotation. DCT features are demonstrated to have better tolerance to image noise than ZM. The results also demonstrate a slight improvement of 5% using SVM as compared to HMM. The second part of this thesis describes a video-based, temporal segmentation framework to detect key frames corresponding to the start and stop of utterances from an image sequence, without using the acoustic signals. This segmentation technique integrates mouth movement and appearance information. The efficacy of this technique was tested through experimental evaluation and satisfactory performance was achieved. This segmentation method has been demonstrated to perform efficiently for utterances separated with short pauses. Potential applications for lip-reading technologies include human computer interface (HCI) for mobility-impaired users, defense applications that require voice-less communication, lip-reading mobile phones, in-vehicle systems, and improvement of speech-based computer control in noisy environments.

<p>This dissertation addresses the challenges of giving computers the ability of doing face recognition, i.e. discriminate between different faces. Face recognition systems are commonly trained with a database of face images, becoming “familiar” with the given faces. Many reported methods rely heavily on training database size and representativenes. But collecting training images covering, for instance, a wide range of viewpoints, different expressions and illumination conditions is difficult and costly. Moreover, there may be only one face image per person at low image resolution or quality. In these situations, face recognition techniques usually suffer serious performance drop. Here we present effective algorithms that deal with single image per person database, despite issues with illumination, face expression and pose variation.</p><p>Illumination changes the appearance of a face in images. Thus, we use a new pyramid based fusion method for face recognition under arbitrary unknown lighting. This extended approach with logarithmic transform works efficiently with a single image. The produced image has better contrast at both low and high ranges, i.e. has more visible details than the original one. An improved method works with high dynamic range images, useful for outdoor face images.</p><p>Face expressions also modify the images’ appearance. An extended Hidden Markov Models (HMM) with a flexible encoding scheme treats images as an ensemble of horizontal and vertical strips. Each person is modeled by Joint Multiple Hidden Markov Models (JM-HMMs). This approach offers computational advantages and the good learning ability from just a single sample per class. A fast method simulated JM-HMM functionality is then derived. The new method with abstract observations and a simplified similarity measurement does not require retraining HMMs for new images or subjects. Pose invariant recognition from a single sample image per person was overcome by using the wire frame Candide face model for the synthesis of virtual views. This is one of the support functions of our face recognition system, WAWO. The extensive experiments clearly show that WAWO outperforms the state-of-the-art systems in FERET tests.</p>

<p>Man portable air defence systems, MANPADS, pose a big threat to civilian and military aircraft. This thesis aims to find methods that could be used in a missile approach warning system based on infrared cameras.</p><p>The two main tasks of the completed system are to classify the type of missile, and also to estimate its position and velocity from a sequence of images.</p><p>The classification is based on hidden Markov models, one-class classifiers, and multi-class classifiers.</p><p>Position and velocity estimation uses a model of the observed intensity as a function of real intensity, image coordinates, distance and missile orientation. The estimation is made by an extended Kalman filter.</p><p>We show that fast classification of missiles based on radiometric data and a hidden Markov model is possible and works well, although more data would be needed to verify the results.</p><p>Estimating the position and velocity works fairly well if the initial parameters are known. Unfortunately, some of these parameters can not be computed using the available sensor data.</p>

This dissertation addresses the challenges of giving computers the ability of doing face recognition, i.e. discriminate between different faces. Face recognition systems are commonly trained with a database of face images, becoming “familiar” with the given faces. Many reported methods rely heavily on training database size and representativenes. But collecting training images covering, for instance, a wide range of viewpoints, different expressions and illumination conditions is difficult and costly. Moreover, there may be only one face image per person at low image resolution or quality. In these situations, face recognition techniques usually suffer serious performance drop. Here we present effective algorithms that deal with single image per person database, despite issues with illumination, face expression and pose variation. Illumination changes the appearance of a face in images. Thus, we use a new pyramid based fusion method for face recognition under arbitrary unknown lighting. This extended approach with logarithmic transform works efficiently with a single image. The produced image has better contrast at both low and high ranges, i.e. has more visible details than the original one. An improved method works with high dynamic range images, useful for outdoor face images. Face expressions also modify the images’ appearance. An extended Hidden Markov Models (HMM) with a flexible encoding scheme treats images as an ensemble of horizontal and vertical strips. Each person is modeled by Joint Multiple Hidden Markov Models (JM-HMMs). This approach offers computational advantages and the good learning ability from just a single sample per class. A fast method simulated JM-HMM functionality is then derived. The new method with abstract observations and a simplified similarity measurement does not require retraining HMMs for new images or subjects. Pose invariant recognition from a single sample image per person was overcome by using the wire frame Candide face model for the synthesis of virtual views. This is one of the support functions of our face recognition system, WAWO. The extensive experiments clearly show that WAWO outperforms the state-of-the-art systems in FERET tests.

In this master's thesis, hidden Markov models (HMM) are evaluated as a tool for forecasting movements in a currency cross. With an ever increasing electronic market, making way for more automated trading, or so called algorithmic trading, there is constantly a need for new trading strategies trying to find alpha, the excess return, in the market. HMMs are based on the well-known theories of Markov chains, but where the states are assumed hidden, governing some observable output. HMMs have mainly been used for speech recognition and communication systems, but have lately also been utilized on financial time series with encouraging results. Both discrete and continuous versions of the model will be tested, as well as single- and multivariate input data. In addition to the basic framework, two extensions are implemented in the belief that they will further improve the prediction capabilities of the HMM. The first is a Gaussian mixture model (GMM), where one for each state assign a set of single Gaussians that are weighted together to replicate the density function of the stochastic process. This opens up for modeling non-normal distributions, which is often assumed for foreign exchange data. The second is an exponentially weighted expectation maximization (EWEM) algorithm, which takes time attenuation in consideration when re-estimating the parameters of the model. This allows for keeping old trends in mind while more recent patterns at the same time are given more attention. Empirical results shows that the HMM using continuous emission probabilities can, for some model settings, generate acceptable returns with Sharpe ratios well over one, whilst the discrete in general performs poorly. The GMM therefore seems to be an highly needed complement to the HMM for functionality. The EWEM however does not improve results as one might have expected. Our general impression is that the predictor using HMMs that we have developed and tested is too unstable to be taken in as a trading tool on foreign exchange data, with too many factors influencing the results. More research and development is called for.

Long terminal repeats (LTRs) are non-coding repeats flanking the protein-coding genes of LTR retrotransposons. The variability of LTRs poses a challenge in studying them. Hidden Markov models (HMMs), probabilistic models widely used in pattern recognition, are useful in dealing with this variability. The aim of this work was mainly to study LTRs of retroviruses and LTR retrotransposons using HMMs. Paper I describes the methodology of HMM modelling applied to different groups of LTRs from exogenous retroviruses (XRVs) and endogenous retroviruses (ERVs). The detection capabilities of HMMs were assessed and were found to be high for homogeneous groups of LTRs. The alignments generated by the HMMs displayed conserved motifs some of which could be related to known functions of XRVs. The common features of the different groups of retroviral LTRs were investigated by combining them into a single alignment. They were the short inverted terminal repeats TG and CA and three AT-rich stretches which provide retroviruses with TATA boxes and AATAAA polyadenylation signals. In Paper II, phylogenetic trees of three groups of retroviral LTRs were constructed by using HMM-based alignments. The LTR trees were consistent with trees based on other retroviral genes suggesting co-evolution between LTRs and these genes. In Paper III, the methods in Paper I and II were extended to LTRs from other retrotransposon groups, covering much of the diversity of all known LTRs. For the first time an LTR phylogeny could be achieved. There were no major disagreement between the LTR tree and trees based on three different domains of the Pol gene. The conserved LTR structure of paper I was found to apply to all LTRs. Putative Integrase recognition motifs extended up to 12 bp beyond the short inverted repeats TG/CA. Paper IV is a review article describing the use of sequence similarity and structural markers for the taxonomy of ERVs. ERVs were originally classified into three classes according to the length of the target site duplication. While this classification is useful it does not include all ERVs. A naming convention based on previous ERV and XRV nomenclature but taking into account newer information is advocated in order to provide a practical yet coherent scheme in dealing with new unclassified ERV sequences. Paper V gives an overview of bioinformatics tools for studies of ERVs and of retroviral evolution before and after endogenization. It gives some examples of recent integrations in vertebrate genomes and discusses pathogenicity of human ERVs including their possible relation to cancers. In conclusion, HMMs were able to successfully detect and align LTRs. Progress was made in understanding their conserved structure and phylogeny. The methods developed in this thesis could be applied to different kinds of non-coding DNA sequence element.

We investigate the unsupervised K-means clustering and the semi-supervised hidden Markov model (HMM) to automatically detect anomalous motion patterns in groups of people (crowds). Anomalous motion patterns are typically people merging into a dense group, followed by disturbances or threatening situations within the group. The application of K-means clustering and HMM are illustrated with datasets from four surveillance scenarios. The results indicate that by investigating the group of people in a systematic way with different K values, analyze cluster density, cluster quality and changes in cluster shape we can automatically detect anomalous motion patterns. The results correspond well with the events in the datasets. The results also indicate that very accurate detections of the people in the dense group would not be necessary. The clustering and HMM results will be very much the same also with some increased uncertainty in the detections. / <p>Funding Agencies|Vinnova (Swedish Governmental Agency for Innovation Systems) under the VINNMER program||</p>

In this thesis, we study the shape analysis problem and propose new methods for shape description, similarity and recognition. Firstly, we introduce a new shape descriptor in a two-step method. In the first step, the 2-D shape information is mapped into a set of 1-D functions. The mapping is based on the beams, which are originated from a boundary point, connecting that point with the rest of the points on the boundary. At each point, the angle between a pair of beams is taken as a random variable to define the statistics of the topological structure of the boundary. The third order statistics of all the beam angles is used to construct 1-D Beam Angle Statistics (BAS) functions. In the second step, we apply a set of feature extraction methods on BAS functions in order to describe it in a more compact form. BAS functions eliminate the context-dependency of the representation to the data set. BAS function is invariant to translation, rotation and scale. It is insensitive to distortions. No predefined resolution or threshold is required to define the BAS functions. Secondly, we adopt three different similarity distance methods defined on the BAS feature space, namely, Optimal Correspondence of String Subsequences, Dynamic Warping and Cyclic Sequence Matching algorithms. Main goal in these algorithms is to minimize the distance between two BAS features by allowing deformations. Thirdly, we propose a new Hidden Markov Model (HMM)topology for boundary based shape recognition. The proposed topology called Circular HMM is both ergodic and temporal. Therefore, the states can be revisited in finite time intervals while keeping the sequential information in the string, which represents the shape. It is insensitive to size changes. Since it has no starting and terminating state, it is insensitive to the starting point of the shape boundary. Experiments are done on the dataset of MPEG 7 Core Experiments Shape-1. It is observed that BAS descriptor outperforms all the methods in the literature. The Circular HMM gives higher recognition rates than the classical topologies in shape analysis applications.

Man portable air defence systems, MANPADS, pose a big threat to civilian and military aircraft. This thesis aims to find methods that could be used in a missile approach warning system based on infrared cameras. The two main tasks of the completed system are to classify the type of missile, and also to estimate its position and velocity from a sequence of images. The classification is based on hidden Markov models, one-class classifiers, and multi-class classifiers. Position and velocity estimation uses a model of the observed intensity as a function of real intensity, image coordinates, distance and missile orientation. The estimation is made by an extended Kalman filter. We show that fast classification of missiles based on radiometric data and a hidden Markov model is possible and works well, although more data would be needed to verify the results. Estimating the position and velocity works fairly well if the initial parameters are known. Unfortunately, some of these parameters can not be computed using the available sensor data.

<p>Hundreds of different factors adorn the eukaryotic genome, binding to it in large number. These DNA binding factors (DBFs) include nucleosomes, transcription factors (TFs), and other proteins and protein complexes, such as the origin recognition complex (ORC). DBFs compete with one another for binding along the genome, yet many current models of genome binding do not consider different types of DBFs together simultaneously. Additionally, binding is a stochastic process that results in a continuum of binding probabilities at any position along the genome, but many current models tend to consider positions as being either binding sites or not.</p><p>Here, we present a model that allows a multitude of DBFs, each at different concentrations, to compete with one another for binding sites along the genome. The result is an 'occupancy profile', a probabilistic description of the DNA occupancy of each factor at each position. We implement our model efficiently as the software package COMPETE. We demonstrate genome-wide and at specific loci how modeling nucleosome binding alters TF binding, and vice versa, and illustrate how factor concentration influences binding occupancy. Binding cooperativity between nearby TFs arises implicitly via mutual competition with nucleosomes. Our method applies not only to TFs, but also recapitulates known occupancy profiles of a well-studied replication origin with and without ORC binding.</p><p>We then develop a statistical framework for tuning our model concentrations to further improve its predictions. Importantly, this tuning optimizes with respect to actual biological data. We take steps to ensure that our tuned parameters are biologically plausible.</p><p>Finally, we discuss novel extensions and applications of our model, suggesting next steps in its development and deployment.</p> / Dissertation