In Natural Language Processing (NLP), speech and text are parsed and generated with language models and parser models, and translated with translation models. Each model contains a set of numerical parameters which are found by applying a suitable training algorithm to a set of training data. Many such training algorithms are instances of the Expectation-Maximization (EM) algorithm. In [BSV15], a generic EM algorithm for NLP is described. This work presents a particular speech model, the Hidden Markov model, and its standard training algorithm, the Baum-Welch algorithm. It is then shown that the Baum-Welch algorithm is an instance of the generic EM algorithm introduced by [BSV15], from which follows that all statements about the generic EM algorithm also apply to the Baum-Welch algorithm, especially its correctness and convergence properties.

No description available.

The valuation of weather derivatives is greatly dependent on accurate modeling and forecasting of the underlying temperature indices. The complexity and uncertainty in such modeling has led to several temperature processes being developed for the Monte Carlo simulation of daily average temperatures. In this report, we aim to compare the results of two recently developed models by Gyamerah et al. (2018) and Evarest, Berntsson, Singull, and Yang (2018). The paper gives a thorough introduction to option theory, Lévy and Wiener processes, and generalized hyperbolic distributions frequently used in temperature modeling. Implementations of maximum likelihood estimation and the expectation-maximization algorithm with Kim's smoothed transition probabilities are used to fit the Lévy process distributions and both models' parameters, respectively. Later, the use of both models is considered for the pricing of European HDD and CDD options by Monte Carlo simulation. The evaluation shows a tendency toward the shifted temperature regime over the base regime, in contrast to the two articles, when evaluated for three data sets. Simulation is successfully demonstrated for the model of Evarest, however Gyamerah's model was unable to be replicated. This is concluded to be due to the two articles containing several incorrect derivations, why the thesis is left unanswered and the articles' conclusions are questioned. We end by proposing further validation of the two models and summarize the alterations required for a correct implementation.

Probabilistic graphical modeling is a framework which can be used to succinctly<br>represent multivariate probability distributions of time series in terms of each time<br>series’s dependence on others. In general, it is computationally prohibitive to sta-<br>tistically infer an arbitrary model from data. However, if we constrain the model to<br>have a tree topology, the corresponding learning algorithms become tractable. The<br>expressive power of tree-structured distributions are low, since only n − 1 dependen-<br>cies are explicitly encoded for an n node tree. One way to improve the expressive<br>power of tree models is to combine many of them in a mixture model. This work<br>presents and uses simulations to validate extensions of the standard mixtures of trees<br>model for i.i.d data to the setting of time series data. We also consider the setting<br>where the tree mixture itself forms a hidden Markov chain, which could be better<br>suited for approximating time-varying seasonal data in the real world. Both of these<br>are evaluated on artificial data sets.<br><br>

In Natural Language Processing (NLP), speech and text are parsed and generated with language models and parser models, and translated with translation models. Each model contains a set of numerical parameters which are found by applying a suitable training algorithm to a set of training data. Many such training algorithms are instances of the Expectation-Maximization (EM) algorithm. In [BSV15], a generic EM algorithm for NLP is described. This work presents a particular speech model, the Hidden Markov model, and its standard training algorithm, the Baum-Welch algorithm. It is then shown that the Baum-Welch algorithm is an instance of the generic EM algorithm introduced by [BSV15], from which follows that all statements about the generic EM algorithm also apply to the Baum-Welch algorithm, especially its correctness and convergence properties.:1 Introduction 1.1 N-gram models 1.2 Hidden Markov model 2 Expectation-maximization algorithms 2.1 Preliminaries 2.2 Algorithmic skeleton 2.3 Corpus-based step mapping 2.4 Simple counting step mapping 2.5 Regular tree grammars 2.6 Inside-outside step mapping 2.7 Review 3 The Hidden Markov model 3.1 Forward and backward algorithms 3.2 The Baum-Welch algorithm 3.3 Deriving the Baum-Welch algorithm 3.3.1 Model parameter and countable events 3.3.2 Tree-shaped hidden information 3.3.3 Complete-data corpus 3.3.4 Inside weights 3.3.5 Outside weights 3.3.6 Complete-data corpus (cont.) 3.3.7 Step mapping 3.4 Review Appendix A Elided proofs from Chapter 3 A.1 Proof of Lemma 3.8 A.2 Proof of Lemma 3.9 B Formulary for Chapter 3 Bibliography

This thesis present a Stochastic Volatility in Mean (SVM) model which is estimated using sequential Monte Carlo methods. The SVM model was first introduced by Koopman and provides an opportunity to study the intertemporal relationship between stock returns and their volatility through inclusion of volatility itself as an explanatory variable in the mean-equation. Using sequential Monte Carlo methods allows us to consider a non-linear estimation procedure at cost of introducing extra computational complexity. The recently developed PaRIS-algorithm, introduced by Olsson and Westerborn, drastically decrease the computational complexity of smoothing relative to previous algorithms and allows for efficient estimation of parameters. The main purpose of this thesis is to investigate the volatility feedback effect, i.e. the relation between expected return and unexpected volatility in an empirical study. The results shows that unanticipated shocks to the return process do not explain expected returns. / Detta examensarbete presenterar en stokastisk volatilitets medelvärdes (SVM) modell som estimeras genom sekventiella Monte Carlo metoder. SVM-modellen introducerades av Koopman och ger en möjlighet att studera den samtida relationen mellan aktiers avkastning och deras volatilitet genom att inkludera volatilitet som en förklarande variabel i medelvärdes-ekvationen. Sekventiella Monte Carlo metoder tillåter oss att använda icke-linjära estimerings procedurer till en kostnad av extra beräkningskomplexitet. Den nyligen utvecklad PaRIS-algoritmen, introducerad av Olsson och Westerborn, minskar drastiskt beräkningskomplexiteten jämfört med tidigare algoritmer och tillåter en effektiv uppskattning av parametrar. Huvudsyftet med detta arbete är att undersöka volatilitets-återkopplings-teorin d.v.s. relationen mellan förväntad avkastning och oväntad volatilitet i en empirisk studie. Resultatet visar på att oväntade chockar i avkastningsprocessen inte har förklarande förmåga över förväntad avkastning.

Nous y introduisons une nouvelle classe de distributions bivariées de type Marshall-Olkin, la distribution Erlang bivariée. La transformée de Laplace, les moments et les densités conditionnelles y sont obtenus. Les applications potentielles en assurance-vie et en finance sont prises en considération. Les estimateurs du maximum de vraisemblance des paramètres sont calculés par l'algorithme Espérance-Maximisation. Ensuite, notre projet de recherche est consacré à l'étude des processus de risque multivariés, qui peuvent être utiles dans l'étude des problèmes de la ruine des compagnies d'assurance avec des classes dépendantes. Nous appliquons les résultats de la théorie des processus de Markov déterministes par morceaux afin d'obtenir les martingales exponentielles, nécessaires pour établir des bornes supérieures calculables pour la probabilité de ruine, dont les expressions sont intraitables. / In this contribution, we introduce a new class of bivariate distributions of Marshall-Olkin type, called bivariate Erlang distributions. The Laplace transform, product moments and conditional densities are derived. Potential applications of bivariate Erlang distributions in life insurance and finance are considered. Further, our research project is devoted to the study of multivariate risk processes, which may be useful in analyzing ruin problems for insurance companies with a portfolio of dependent classes of business. We apply results from the theory of piecewise deterministic Markov processes in order to derive exponential martingales needed to establish computable upper bounds of the ruin probabilities, as their exact expressions are intractable.

Cette thèse traite de l’identification de systèmes dynamiques à partir de données échantillonnées à pas variable. Ce type de données est souvent rencontré dans les domaines biomédical, environnemental, dans le cas des systèmes mécaniques où un échantillonnage angulaire est réalisé ou lorsque les données transitent sur un réseau. L’identification directe de modèles à temps continu est l’approche à privilégier lorsque les données disponibles sont échantillonnées à pas variable ; les paramètres des modèles à temps discret étant dépendants de la période d’échantillonnage. Dans une première partie, un estimateur optimal de type variable instrumentale est développé pour estimer les paramètres d’un modèle Box-Jenkins à temps continu. Ce dernier est itératif et présente l’avantage de fournir des estimées non biaisées lorsque le bruit de mesure est coloré et sa convergence est peu sensible au choix du vecteur de paramètres initial. Une difficulté majeure dans le cas où les données sont échantillonnées à pas variable concerne l’estimation de modèles de bruit de type AR et ARMA à temps continu (CAR et CARMA). Plusieurs estimateurs pour les modèles CAR et CARMA s’appuyant sur l’algorithme Espérance-Maximisation (EM) sont développés puis inclus dans l’estimateur complet de variable instrumentale optimale. Une version étendue au cas de l’identification en boucle fermée est également développée. Dans la deuxième partie de la thèse, un estimateur robuste pour l'identification de systèmes à retard est proposé. Cette classe de systèmes est très largement rencontrée en pratique et les méthodes disponibles ne peuvent pas traiter le cas de données échantillonnées à pas variable. Le retard n’est pas contraint à être un multiple de la période d’échantillonnage, contrairement à l’hypothèse traditionnelle dans le cas de modèles à temps discret. L’estimateur développé est de type bootstrap et combine la méthode de variable instrumentale itérative pour les paramètres de la fonction de transfert avec un algorithme numérique de type gradient pour estimer le retard. Un filtrage de type passe-bas est introduit pour élargir la région de convergence pour l’estimation du retard. Tous les estimateurs proposés sont inclus dans la boîte à outils logicielle CONTSID pour Matlab et sont évalués à l’aide de simulation de Monte-Carlo / The output of a system is always corrupted by additive noise, therefore it is more practical to develop estimation algorithms that are capable of handling noisy data. The effect of white additive noise has been widely studied, while a colored additive noise attracts less attention, especially for a continuous-time (CT) noise. Sampling issues of CT stochastic processes are reviewed in this thesis, several sampling schemes are presented. Estimation of a CT stochastic process is studied. An expectation-maximization-based (EM) method to CT autoregressive/autoregressive moving average model is developed, which gives accurate estimation over a large range of sampling interval. Estimation of CT Box-Jenkins models is also considered in this thesis, in which the noise part is modeled to improve the performance of plant model estimation. The proposed method for CT Box-Jenkins model identification is in a two-step and iterative framework. Two-step means the plant and noise models are estimated in a separate and alternate way, where in estimating each of them, the other is assumed to be fixed. More specifically, the plant is estimated by refined instrumental variable (RIV) method while the noise is estimated by EM algorithm. Iterative means that the proposed method repeats the estimation procedure several times until a optimal estimate is found. Many practical systems have inherent time-delay. The problem of identifying delayed systems are of great importance for analysis, prediction or control design. The presence of a unknown time-delay greatly complicates the parameter estimation problem, essentially because the model are not linear with respect to the time-delay. An approach to continuous-time model identification of time-delay systems, combining a numerical search algorithm for the delay with the RIV method for the dynamic has been developed in this thesis. In the proposed algorithm, the system parameters and time-delay are estimated reciprocally in a bootstrap manner. The time-delay is estimated by an adaptive gradient-based method, whereas the system parameters are estimated by the RIV method. Since numerical method is used in this algorithm, the bootstrap method is likely to converge to local optima, therefore a low-pass filter has been used to enlarge the convergence region for the time-delay. The performance of the proposed algorithms are evaluated by numerical examples

This thesis addresses a challenging problem: “how to let the visually impaired ‘see’ others emotions”. We, human beings, are heavily dependent on facial expressions to express ourselves. A smile shows that the person you are talking to is pleased, amused, relieved etc. People use emotional information from facial expressions to switch between conversation topics and to determine attitudes of individuals. Missing emotional information from facial expressions and head gestures makes the visually impaired extremely difficult to interact with others in social events. To enhance the visually impaired’s social interactive ability, in this thesis we have been working on the scientific topic of ‘expressing human emotions through vibrotactile patterns’. It is quite challenging to deliver human emotions through touch since our touch channel is very limited. We first investigated how to render emotions through a vibrator. We developed a real time “lipless” tracking system to extract dynamic emotions from the mouth and employed mobile phones as a platform for the visually impaired to perceive primary emotion types. Later on, we extended the system to render more general dynamic media signals: for example, render live football games through vibration in the mobile for improving mobile user communication and entertainment experience. To display more natural emotions (i.e. emotion type plus emotion intensity), we developed the technology to enable the visually impaired to directly interpret human emotions. This was achieved by use of machine vision techniques and vibrotactile display. The display is comprised of a ‘vibration actuators matrix’ mounted on the back of a chair and the actuators are sequentially activated to provide dynamic emotional information. The research focus has been on finding a global, analytical, and semantic representation for facial expressions to replace state of the art facial action coding systems (FACS) approach. We proposed to use the manifold of facial expressions to characterize dynamic emotions. The basic emotional expressions with increasing intensity become curves on the manifold extended from the center. The blends of emotions lie between those curves, which could be defined analytically by the positions of the main curves. The manifold is the “Braille Code” of emotions. The developed methodology and technology has been extended for building assistive wheelchair systems to aid a specific group of disabled people, cerebral palsy or stroke patients (i.e. lacking fine motor control skills), who don’t have ability to access and control the wheelchair with conventional means, such as joystick or chin stick. The solution is to extract the manifold of the head or the tongue gestures for controlling the wheelchair. The manifold is rendered by a 2D vibration array to provide user of the wheelchair with action information from gestures and system status information, which is very important in enhancing usability of such an assistive system. Current research work not only provides a foundation stone for vibrotactile rendering system based on object localization but also a concrete step to a new dimension of human-machine interaction. / Taktil Video

退休金是退休人未來生活的依靠，確保在退休後能得到適足的退休給付，政府在退休金上實施保證收益制度，此制度為最低保證利率與投資報酬率連結。本文探討退休金給付標準為確定提撥制，當退休金的投資報酬率是根據其連結之股價指數的表現來計算時，股價指數報酬率的模型假設為馬可夫調控跳躍過程模型，考慮市場狀態與布朗運動項、跳躍項的跳躍頻率相關，即為Elliot et al. (2007) 的模型特例。使用1999年至2012年的道瓊工業指數與S&P 500指數的股價指數對數報酬率作為研究資料，採用EM演算法估計參數及SEM演算法估計參數共變異數矩陣。透過概似比檢定說明馬可夫調控跳躍過程模型比狀態轉換模型、跳躍風險下狀態轉換模型更適合描述股價指數報酬率變動情形，也驗證馬可夫調控跳躍過程模型具有描述報酬率不對稱、高狹峰及波動叢聚的特性。最後，假設最低保證利率為固定下，利用Esscher轉換法計算不同模型下型I保證之確定提撥制退休金的評價公式，從公式中可看出受雇人提領的退休金價值可分為政府補助與個人帳戶擁有之退休金兩部分。以執行敏感度分析探討估計參數對於馬可夫調控跳躍過程模型評價公式的影響，而型II保證之確定提撥制退休金的價值則以蒙地卡羅法模擬並探討其敏感度分析結果。 / Pension plan make people a guarantee life in their retirement. In order to ensure the appropriate amount of pension plan, government guarantees associated with pension plan which ties minimum rate of return guarantees and underlying asset rate of return. In this paper, we discussed the pension plan with defined contribution (DC). When the return of asset is based on the stock indices, the return model was set on the assumption that markov-modulated jump diffusion model (MMJDM) could the Brownian motion term and jump rate be both related to market states. This model is the specific case of Elliot et al. (2007) offering. The sample observations is Dow-Jones industrial average and S&P 500 index from 1999 to 2012 by logarithm return of the stock indices. We estimated the parameters by the Expectation-Maximization (EM) algorithm and calculated the covariance matrix of the estimates by supplemented EM (SEM) algorithm. Through the likelihood ratio test (LRT), the data fitted the MMJDM better than other models. The empirical evidence indicated that the MMJDM could describe the asset return for asymmetric, leptokurtic, volatility clustering particularly. Finally, we derived different model's valuation formula for DC pension plan with type-I guarantee by Esscher transformation under rate of return guarantees is constant. From the formula, the value of the pension plan could divide into two segment: government supplement and employees deposit made pension to their personal bank account. And then, we done sensitivity analysis through the MMJDM valuation formula. We used Monte Carlo simulations to evaluate the valuation of DC pension plan with type-II guarantee and discussed it from sensitivity analysis.