In Natural Language Processing (NLP), speech and text are parsed and generated with language models and parser models, and translated with translation models. Each model contains a set of numerical parameters which are found by applying a suitable training algorithm to a set of training data. Many such training algorithms are instances of the Expectation-Maximization (EM) algorithm. In [BSV15], a generic EM algorithm for NLP is described. This work presents a particular speech model, the Hidden Markov model, and its standard training algorithm, the Baum-Welch algorithm. It is then shown that the Baum-Welch algorithm is an instance of the generic EM algorithm introduced by [BSV15], from which follows that all statements about the generic EM algorithm also apply to the Baum-Welch algorithm, especially its correctness and convergence properties.:1 Introduction 1.1 N-gram models 1.2 Hidden Markov model 2 Expectation-maximization algorithms 2.1 Preliminaries 2.2 Algorithmic skeleton 2.3 Corpus-based step mapping 2.4 Simple counting step mapping 2.5 Regular tree grammars 2.6 Inside-outside step mapping 2.7 Review 3 The Hidden Markov model 3.1 Forward and backward algorithms 3.2 The Baum-Welch algorithm 3.3 Deriving the Baum-Welch algorithm 3.3.1 Model parameter and countable events 3.3.2 Tree-shaped hidden information 3.3.3 Complete-data corpus 3.3.4 Inside weights 3.3.5 Outside weights 3.3.6 Complete-data corpus (cont.) 3.3.7 Step mapping 3.4 Review Appendix A Elided proofs from Chapter 3 A.1 Proof of Lemma 3.8 A.2 Proof of Lemma 3.9 B Formulary for Chapter 3 Bibliography

In this thesis, we develop some inferential results for one-shot device testing data analysis. These extend and generalize existing methods in the literature. First, a competing-risk model is introduced for one-shot testing data under accelerated life-tests. One-shot devices are products which will be destroyed immediately after use. Therefore, we can observe only a binary status as data, success or failure, of such products instead of its lifetime. Many one-shot devices contain multiple components and failure of any one of them will lead to the failure of the device. Failed devices are inspected to identify the specific cause of failure. Since the exact lifetime is not observed, EM algorithm becomes a natural tool to obtain the maximum likelihood estimates of the model parameters. Here, we develop the EM algorithm for competing exponential and Weibull cases. Second, a semi-parametric approach is developed for simple one-shot device testing data. Semi-parametric estimation is a model that consists of parametric and non-parametric components. For this purpose, we only assume the hazards at different stress levels are proportional to each other, but no distributional assumption is made on the lifetimes. This provides a greater flexibility in model fitting and enables us to examine the relationship between the reliability of devices and the stress factors. Third, Bayesian inference is developed for one-shot device testing data under exponential distribution and Weibull distribution with non-constant shape parameters for competing risks. Bayesian framework provides statistical inference from another perspective. It assumes the model parameters to be random and then improves the inference by incorporating expert's experience as prior information. This method is shown to be very useful if we have limited failure observation wherein the maximum likelihood estimator may not exist. The thesis proceeds as follows. In Chapter 2, we assume the one-shot devices to have two components with lifetimes having exponential distributions with multiple stress factors. We then develop an EM algorithm for developing likelihood inference for the model parameters as well as some useful reliability characteristics. In Chapter 3, we generalize to the situation when lifetimes follow a Weibull distribution with non-constant shape parameters. In Chapter 4, we propose a semi-parametric model for simple one-shot device test data based on proportional hazards model and develop associated inferential results. In Chapter 5, we consider the competing risk model with exponential lifetimes and develop inference by adopting the Bayesian approach. In Chapter 6, we generalize these results on Bayesian inference to the situation when the lifetimes have a Weibull distribution. Finally, we provide some concluding remarks and indicate some future research directions in Chapter 7. / Thesis / Doctor of Philosophy (PhD)

No description available.

In this work, we propose a novel two-component mixture model: the first component is the three-parameter generalized Gaussian distribution (GGD), and the second is a new three-parameter family of positive densities on the real line. The novelty of our mixture model is that we allow the two components to have totally different parametric families of distributions with asymmetric tails of the mixture density. We extend the scale invariant variable fractional moments (SIVFM) method proposed by Song for the GGD to the parameter estimation of our mixture model. We show that the SIVFM population and sample equations for the second component share very similar desirable global properties such as convexity and unique global roots as those for the GGD given in earlier research. The two-component mixing of these properties make the SIVFM mixture population and estimation equations well-behaved resulting in easy to compute estimators without the issue with starting values. The asymptotic results such as consistency and limiting distribution of the estimators are presented. Furthermore, SIVFM estimators can also serve as a consistent initial estimator for the EM algorithm leading to improved accuracy of the EM algorithm. These algorithms are applied to the analysis of the average amount of precipitation (rainfall) for each of 70 United States (and Puerto Rican) cities clearly demonstrating the bimodal distribution of the estimated mixture density.

Le modèle GARCH à changement de régimes est le fondement de cette thèse. Ce modèle offre de riches dynamiques pour modéliser les données financières en combinant une structure GARCH avec des paramètres qui varient dans le temps. Cette flexibilité donne malheureusement lieu à un problème de path dependence, qui a empêché l'estimation du modèle par le maximum de vraisemblance depuis son introduction, il y a déjà près de 20 ans. La première moitié de cette thèse procure une solution à ce problème en développant deux méthodologies permettant de calculer l'estimateur du maximum de vraisemblance du modèle GARCH à changement de régimes. La première technique d'estimation proposée est basée sur l'algorithme Monte Carlo EM et sur l'échantillonnage préférentiel, tandis que la deuxième consiste en la généralisation des approximations du modèle introduites dans les deux dernières décennies, connues sous le nom de collapsing procedures. Cette généralisation permet d'établir un lien méthodologique entre ces approximations et le filtre particulaire. La découverte de cette relation est importante, car elle permet de justifier la validité de l'approche dite par collapsing pour estimer le modèle GARCH à changement de régimes. La deuxième moitié de cette thèse tire sa motivation de la crise financière de la fin des années 2000 pendant laquelle une mauvaise évaluation des risques au sein de plusieurs compagnies financières a entraîné de nombreux échecs institutionnels. À l'aide d'un large éventail de 78 modèles économétriques, dont plusieurs généralisations du modèle GARCH à changement de régimes, il est démontré que le risque de modèle joue un rôle très important dans l'évaluation et la gestion du risque d'investissement à long terme dans le cadre des fonds distincts. Bien que la littérature financière a dévoué beaucoup de recherche pour faire progresser les modèles économétriques dans le but d'améliorer la tarification et la couverture des produits financiers, les approches permettant de mesurer l'efficacité d'une stratégie de couverture dynamique ont peu évolué. Cette thèse offre une contribution méthodologique dans ce domaine en proposant un cadre statistique, basé sur la régression, permettant de mieux mesurer cette efficacité. / The Markov-switching GARCH model is the foundation of this thesis. This model offers rich dynamics to model financial data by allowing for a GARCH structure with time-varying parameters. This flexibility is unfortunately undermined by a path dependence problem which has prevented maximum likelihood estimation of this model since its introduction, almost 20 years ago. The first half of this thesis provides a solution to this problem by developing two original estimation approaches allowing us to calculate the maximum likelihood estimator of the Markov-switching GARCH model. The first method is based on both the Monte Carlo expectation-maximization algorithm and importance sampling, while the second consists of a generalization of previously proposed approximations of the model, known as collapsing procedures. This generalization establishes a novel relationship in the econometric literature between particle filtering and collapsing procedures. The discovery of this relationship is important because it provides the missing link needed to justify the validity of the collapsing approach for estimating the Markov-switching GARCH model. The second half of this thesis is motivated by the events of the financial crisis of the late 2000s during which numerous institutional failures occurred because risk exposures were inappropriately measured. Using 78 different econometric models, including many generalizations of the Markov-switching GARCH model, it is shown that model risk plays an important role in the measurement and management of long-term investment risk in the context of variable annuities. Although the finance literature has devoted a lot of research into the development of advanced models for improving pricing and hedging performance, the approaches for measuring dynamic hedging effectiveness have evolved little. This thesis offers a methodological contribution in this area by proposing a statistical framework, based on regression analysis, for measuring the effectiveness of dynamic hedges for long-term investment guarantees.

Les simulations ont été implémentées avec le programme Java. / Le regroupement des données est une méthode classique pour analyser les matrices d'expression génétiques. Lorsque le regroupement est appliqué sur les lignes (gènes), chaque colonne (conditions expérimentales) appartient à toutes les grappes obtenues. Cependant, il est souvent observé que des sous-groupes de gènes sont seulement co-régulés (i.e. avec les expressions similaires) sous un sous-groupe de conditions. Ainsi, les techniques de bi-regroupement ont été proposées pour révéler ces sous-matrices des gènes et conditions. Un bi-regroupement est donc un regroupement simultané des lignes et des colonnes d'une matrice de données. La plupart des algorithmes de bi-regroupement proposés dans la littérature n'ont pas de fondement statistique. Cependant, il est intéressant de porter une attention sur les modèles sous-jacents à ces algorithmes et de développer des modèles statistiques permettant d'obtenir des bi-grappes significatives. Dans cette thèse, nous faisons une revue de littérature sur les algorithmes qui semblent être les plus populaires. Nous groupons ces algorithmes en fonction du type d'homogénéité dans la bi-grappe et du type d'imbrication que l'on peut rencontrer. Nous mettons en lumière les modèles statistiques qui peuvent justifier ces algorithmes. Il s'avère que certaines techniques peuvent être justifiées dans un contexte bayésien. Nous développons une extension du modèle à carreaux (plaid) de bi-regroupement dans un cadre bayésien et nous proposons une mesure de la complexité du bi-regroupement. Le critère d'information de déviance (DIC) est utilisé pour choisir le nombre de bi-grappes. Les études sur les données d'expression génétiques et les données simulées ont produit des résultats satisfaisants. À notre connaissance, les algorithmes de bi-regroupement supposent que les gènes et les conditions expérimentales sont des entités indépendantes. Ces algorithmes n'incorporent pas de l'information biologique a priori que l'on peut avoir sur les gènes et les conditions. Nous introduisons un nouveau modèle bayésien à carreaux pour les données d'expression génétique qui intègre les connaissances biologiques et prend en compte l'interaction par paires entre les gènes et entre les conditions à travers un champ de Gibbs. La dépendance entre ces entités est faite à partir des graphes relationnels, l'un pour les gènes et l'autre pour les conditions. Le graphe des gènes et celui des conditions sont construits par les k-voisins les plus proches et permet de définir la distribution a priori des étiquettes comme des modèles auto-logistiques. Les similarités des gènes se calculent en utilisant l'ontologie des gènes (GO). L'estimation est faite par une procédure hybride qui mixe les MCMC avec une variante de l'algorithme de Wang-Landau. Les expériences sur les données simulées et réelles montrent la performance de notre approche. Il est à noter qu'il peut exister plusieurs variables de bruit dans les données à micro-puces, c'est-à-dire des variables qui ne sont pas capables de discriminer les groupes. Ces variables peuvent masquer la vraie structure du regroupement. Nous proposons un modèle inspiré de celui à carreaux qui, simultanément retrouve la vraie structure de regroupement et identifie les variables discriminantes. Ce problème est traité en utilisant un vecteur latent binaire, donc l'estimation est obtenue via l'algorithme EM de Monte Carlo. L'importance échantillonnale est utilisée pour réduire le coût computationnel de l'échantillonnage Monte Carlo à chaque étape de l'algorithme EM. Nous proposons un nouveau modèle pour résoudre le problème. Il suppose une superposition additive des grappes, c'est-à-dire qu'une observation peut être expliquée par plus d'une seule grappe. Les exemples numériques démontrent l'utilité de nos méthodes en terme de sélection de variables et de regroupement. / Clustering is a classical method to analyse gene expression data. When applied to the rows (e.g. genes), each column belongs to all clusters. However, it is often observed that the genes of a subset of genes are co-regulated and co-expressed in a subset of conditions, but behave almost independently under other conditions. For these reasons, biclustering techniques have been proposed to look for sub-matrices of a data matrix. Biclustering is a simultaneous clustering of rows and columns of a data matrix. Most of the biclustering algorithms proposed in the literature have no statistical foundation. It is interesting to pay attention to the underlying models of these algorithms and develop statistical models to obtain significant biclusters. In this thesis, we review some biclustering algorithms that seem to be most popular. We group these algorithms in accordance to the type of homogeneity in the bicluster and the type of overlapping that may be encountered. We shed light on statistical models that can justify these algorithms. It turns out that some techniques can be justified in a Bayesian framework. We develop an extension of the biclustering plaid model in a Bayesian framework and we propose a measure of complexity for biclustering. The deviance information criterion (DIC) is used to select the number of biclusters. Studies on gene expression data and simulated data give satisfactory results. To our knowledge, the biclustering algorithms assume that genes and experimental conditions are independent entities. These algorithms do not incorporate prior biological information that could be available on genes and conditions. We introduce a new Bayesian plaid model for gene expression data which integrates biological knowledge and takes into account the pairwise interactions between genes and between conditions via a Gibbs field. Dependence between these entities is made from relational graphs, one for genes and another for conditions. The graph of the genes and conditions is constructed by the k-nearest neighbors and allows to define a priori distribution of labels as auto-logistic models. The similarities of genes are calculated using gene ontology (GO). To estimate the parameters, we adopt a hybrid procedure that mixes MCMC with a variant of the Wang-Landau algorithm. Experiments on simulated and real data show the performance of our approach. It should be noted that there may be several variables of noise in microarray data. These variables may mask the true structure of the clustering. Inspired by the plaid model, we propose a model that simultaneously finds the true clustering structure and identifies discriminating variables. We propose a new model to solve the problem. It assumes that an observation can be explained by more than one cluster. This problem is addressed by using a binary latent vector, so the estimation is obtained via the Monte Carlo EM algorithm. Importance Sampling is used to reduce the computational cost of the Monte Carlo sampling at each step of the EM algorithm. Numerical examples demonstrate the usefulness of these methods in terms of variable selection and clustering.

Le modèle GARCH à changement de régimes est le fondement de cette thèse. Ce modèle offre de riches dynamiques pour modéliser les données financières en combinant une structure GARCH avec des paramètres qui varient dans le temps. Cette flexibilité donne malheureusement lieu à un problème de path dependence, qui a empêché l'estimation du modèle par le maximum de vraisemblance depuis son introduction, il y a déjà près de 20 ans. La première moitié de cette thèse procure une solution à ce problème en développant deux méthodologies permettant de calculer l'estimateur du maximum de vraisemblance du modèle GARCH à changement de régimes. La première technique d'estimation proposée est basée sur l'algorithme Monte Carlo EM et sur l'échantillonnage préférentiel, tandis que la deuxième consiste en la généralisation des approximations du modèle introduites dans les deux dernières décennies, connues sous le nom de collapsing procedures. Cette généralisation permet d'établir un lien méthodologique entre ces approximations et le filtre particulaire. La découverte de cette relation est importante, car elle permet de justifier la validité de l'approche dite par collapsing pour estimer le modèle GARCH à changement de régimes. La deuxième moitié de cette thèse tire sa motivation de la crise financière de la fin des années 2000 pendant laquelle une mauvaise évaluation des risques au sein de plusieurs compagnies financières a entraîné de nombreux échecs institutionnels. À l'aide d'un large éventail de 78 modèles économétriques, dont plusieurs généralisations du modèle GARCH à changement de régimes, il est démontré que le risque de modèle joue un rôle très important dans l'évaluation et la gestion du risque d'investissement à long terme dans le cadre des fonds distincts. Bien que la littérature financière a dévoué beaucoup de recherche pour faire progresser les modèles économétriques dans le but d'améliorer la tarification et la couverture des produits financiers, les approches permettant de mesurer l'efficacité d'une stratégie de couverture dynamique ont peu évolué. Cette thèse offre une contribution méthodologique dans ce domaine en proposant un cadre statistique, basé sur la régression, permettant de mieux mesurer cette efficacité. / The Markov-switching GARCH model is the foundation of this thesis. This model offers rich dynamics to model financial data by allowing for a GARCH structure with time-varying parameters. This flexibility is unfortunately undermined by a path dependence problem which has prevented maximum likelihood estimation of this model since its introduction, almost 20 years ago. The first half of this thesis provides a solution to this problem by developing two original estimation approaches allowing us to calculate the maximum likelihood estimator of the Markov-switching GARCH model. The first method is based on both the Monte Carlo expectation-maximization algorithm and importance sampling, while the second consists of a generalization of previously proposed approximations of the model, known as collapsing procedures. This generalization establishes a novel relationship in the econometric literature between particle filtering and collapsing procedures. The discovery of this relationship is important because it provides the missing link needed to justify the validity of the collapsing approach for estimating the Markov-switching GARCH model. The second half of this thesis is motivated by the events of the financial crisis of the late 2000s during which numerous institutional failures occurred because risk exposures were inappropriately measured. Using 78 different econometric models, including many generalizations of the Markov-switching GARCH model, it is shown that model risk plays an important role in the measurement and management of long-term investment risk in the context of variable annuities. Although the finance literature has devoted a lot of research into the development of advanced models for improving pricing and hedging performance, the approaches for measuring dynamic hedging effectiveness have evolved little. This thesis offers a methodological contribution in this area by proposing a statistical framework, based on regression analysis, for measuring the effectiveness of dynamic hedges for long-term investment guarantees.

Cette thèse concerne la modélisation de la dynamique des courbes des taux internationales avec prise en compte de plusieurs canaux de dépendance. A l’aide d’une nouvelle base de données des taux souverains internationaux, nous observons que le critère de la variabilité expliquée, proposé par la littérature, n’est pas capable de sélectionner une meilleure combinaison des facteurs décrivant la dynamique jointe des courbes des taux. Nous proposons une méthode nouvelle de section des facteurs fondée sur la maximisation de vraisemblance d’un modèle espace-état linéaire gaussien avec facteurs communs et locaux. Le problème d’identification associée est résolu d’une façon novatrice. En estimant différents combinaisons de pays, nous sélectionnons des deux facteurs globaux et trois locaux ayant un pouvoir prédictif des variables macro-économiques (activité économique et taux d’inflation) dans chaque économie considérée. Notre méthode nous permet aussi de détecter des facteurs cachés dans les rendements obligataires. Ils ne sont pas visibles à travers une analyse classique en composant principales des rendements obligataires et ils contribuent à la prévision du taux d’inflation et du taux de croissance de la production industrielle. / The aim of this thesis is to model the dynamics of international term structure of interest rates taking into consideration several dependence channels.Thanks to a new international Treasury yield curve database, we observe that the explained variability decision criterion, suggested by the literature, is not able to select the best combination of factors characterizing the joint dynamics of yield curves. We propose a new methodology based on the maximisation of the likelihood function of a Gaussian state-space model with common and local factors. The associated identification problem is solved in an innovative way. By estimating several sets of countries, we select two global (and three local) factors which are also useful to forecast macroeconomic variables in each considered economy.In addition, our method allows us to detect hidden factors in the international bond returns. They are not visible through a classical principal component analysis of expected bond returns but they are helpful to forecast inflation and industrial production. Keywords: International treasury yield curves, common and local factors, state-space models, EM algorithm, International bond risk premia, principal components.

The aim of this work is to present an inference and diagnostic study of an extension of the lifetime distribution family proposed by Birnbaum and Saunders (1969a,b). This extension is obtained by considering a skew-elliptical distribution instead of the normal distribution. Specifically, in this work we develop a Birnbaum-Saunders (BS) distribution type based on scale mixtures of skew-normal distributions (SMSN). The resulting family of lifetime distributions represents a robust extension of the usual BS distribution. Based on this family, we reproduce the usual properties of the BS distribution, and present an estimation method based on the EM algorithm. In addition, we present regression models associated with the BS distributions (based on scale mixtures of skew-normal), which are developed as an extension of the sinh-normal distribution (Rieck and Nedelman, 1991). For this model we consider an estimation and diagnostic study for uncensored data. / O objetivo deste trabalho é apresentar um estudo de inferência e diagnóstico em uma extensão da família de distribuições de tempos de vida proposta por Birnbaum e Saunders (1969a,b). Esta extensão é obtida ao considerar uma distribuição skew-elíptica em lugar da distribuição normal. Especificamente, neste trabalho desenvolveremos um tipo de distribuição Birnbaum-Saunders (BS) baseda nas distribuições mistura de escala skew-normal (MESN). Esta família resultante de distribuições de tempos de vida representa uma extensão robusta da distribuição BS usual. Baseado nesta família, vamos reproduzir as propriedades usuais da distribuição BS, e apresentar um método de estimação baseado no algoritmo EM. Além disso, vamos apresentar modelos de regressão associado à distribuições BS (baseada na distribuição mistura de escala skew-normal), que é desenvolvida como uma extensão da distribuição senh-normal (Rieck e Nedelman, 1991), para estes vamos considerar um estudo de estimação e diagnóstisco para dados sem censura.

The aim of this work is to present an inference and diagnostic study of an extension of the lifetime distribution family proposed by Birnbaum and Saunders (1969a,b). This extension is obtained by considering a skew-elliptical distribution instead of the normal distribution. Specifically, in this work we develop a Birnbaum-Saunders (BS) distribution type based on scale mixtures of skew-normal distributions (SMSN). The resulting family of lifetime distributions represents a robust extension of the usual BS distribution. Based on this family, we reproduce the usual properties of the BS distribution, and present an estimation method based on the EM algorithm. In addition, we present regression models associated with the BS distributions (based on scale mixtures of skew-normal), which are developed as an extension of the sinh-normal distribution (Rieck and Nedelman, 1991). For this model we consider an estimation and diagnostic study for uncensored data. / O objetivo deste trabalho é apresentar um estudo de inferência e diagnóstico em uma extensão da família de distribuições de tempos de vida proposta por Birnbaum e Saunders (1969a,b). Esta extensão é obtida ao considerar uma distribuição skew-elíptica em lugar da distribuição normal. Especificamente, neste trabalho desenvolveremos um tipo de distribuição Birnbaum-Saunders (BS) baseda nas distribuições mistura de escala skew-normal (MESN). Esta família resultante de distribuições de tempos de vida representa uma extensão robusta da distribuição BS usual. Baseado nesta família, vamos reproduzir as propriedades usuais da distribuição BS, e apresentar um método de estimação baseado no algoritmo EM. Além disso, vamos apresentar modelos de regressão associado à distribuições BS (baseada na distribuição mistura de escala skew-normal), que é desenvolvida como uma extensão da distribuição senh-normal (Rieck e Nedelman, 1991), para estes vamos considerar um estudo de estimação e diagnóstisco para dados sem censura.