In this project we investigate the susceptibility ofreinforcement rearning (RL) algorithms to adversarial attacks.Adversarial attacks have been proven to be very effective atreducing performance of deep learning classifiers, and recently,have also been shown to reduce performance of RL agents.The goal of this project is to evaluate adversarial attacks onagents trained using deep reinforcement learning (DRL), aswell as to investigate how to detect these types of attacks. Wefirst use DRL to solve two environments from OpenAI’s gymmodule, namely Cartpole and Lunarlander, by using DQN andDDPG (DRL techniques). We then evaluate the performanceof attacks and finally we also train neural networks to detectattacks. The attacks was successful at reducing performancein the LunarLander environment and CartPole environment.The attack detector was very successful at detecting attacks onthe CartPole environment, but performed not quiet as well onLunarLander.We hypothesize that continuous action space environmentsmay pose a greater difficulty for attack detectors to identifypotential adversarial attacks. / I detta projekt undersöker vikänsligheten hos förstärknings lärda (RL) algotritmerför attacker mot förstärknings lärda agenter. Attackermot förstärknings lärda agenter har visat sig varamycket effektiva för att minska prestandan hos djuptförsärknings lärda klassifierare och har nyligen visat sigockså minska prestandan hos förstärknings lärda agenter.Målet med detta projekt är att utvärdera attacker motdjupt förstärknings lärda agenter och försöka utföraoch upptäcka attacker. Vi använder först RL för attlösa två miljöer från OpenAIs gym module CartPole-v0och ContiniousLunarLander-v0 med DQN och DDPG.Vi utvärderar sedan utförandet av attacker och avslutarslutligen med ett möjligt sätt att upptäcka attacker.Attackerna var mycket framgångsrika i att minskaprestandan i både CartPole-miljön och LunarLandermiljön. Attackdetektorn var mycket framgångsrik medatt upptäcka attacker i CartPole-miljön men presteradeinte lika bra i LunarLander-miljön.Vi hypotiserar att miljöer med kontinuerligahandlingsrum kan innebära en större svårighet fören attack identifierare att upptäcka attacker mot djuptförstärknings lärda agenter. / Kandidatexjobb i elektroteknik 2021, KTH, Stockholm

Rieck & Nedelman (1991) propuseram um modelo de regressão log-linear tendo como base a distribuição Birnbaum-Saunders (Birnbaum & Saunders, 1969a). O modelo proposto pelos autores vem sendo bastante explorado e tem se mostrado uma ótima alternativa a outros modelos propostos na literatura, como por exemplo, os modelos de regressão Weibull, gama e lognormal. No entanto, até o presente momento, não existe nenhum estudo tratando de refinamentos para as estatísticas da razão de verossimilhanças e escore nesta classe de modelos de regressão. Assim, um dos objetivos desta tese é obter um fator de correção de Bartlett para a estatística da razão de verossimilhanças e um fator de correção tipo-Bartlett para a estatística escore nesse modelo. Estes ajustes melhoram a aproximação da distribuição nula destas estatísticas pela distribuição qui-quadrado de referência. Adicionalmente, objetiva-se obter ajustes para a estatística da razão de verossimilhanças sinalizada. Tais ajustes melhoram a aproximação desta estatística pela distribuição normal padrão. Recentemente, uma nova estatística de teste foi proposta por Terrell (2002), a qual o autor denomina estatística gradiente. Esta estatística foi derivada a partir da estatística escore e da estatística de Wald modificada (Hayakawa & Puri, 1985). A combinação daquelas duas estatísticas resulta em uma estatística muito simples de ser calculada, não envolvendo, por exemplo, nenhum cálculo matricial como produto e inversa de matrizes. Esta estatística foi recentemente citada por Rao (2005): \"The suggestion by Terrell is attractive as it is simple to compute. It would be of interest to investigate the performance of the [gradient] statistic.\" Caminhando na direção da sugestão de Rao, outro objetivo da tese é obter uma expansão assintótica para a distribuição da estatística gradiente sob uma sequência de alternativas de Pitman convergindo para a hipótese nula a uma taxa de convergência de n^{-1/2} utilizando a metodologia desenvolvida por Peers (1971) e Hayakawa (1975). Em particular, mostramos que, até ordem n^{-1/2}, a estatística gradiente segue distribuição qui-quadrado central sob a hipótese nula e distribuição qui-quadrado não central sob a hipótese alternativa. Também temos como objetivo comparar o poder local deste teste com o poder local dos testes da razão de verossimilhanças, de Wald e escore. Finalmente, aplicaremos a expansão assintótica derivada na tese em algumas classes particulares de modelos. / The Birnbaum-Saunders regression model is commonly used in reliability studies.We address the issue of performing inference in this class of models when the number of observations is small. Our simulation results suggest that the likelihood ratio and score tests tend to be liberal when the sample size is small. We derive Bartlett and Bartlett-type correction factors which reduce the size distortion of the tests. Additionally, we also consider modified signed log-likelihood ratio statistics in this class of models. Finally, the asymptotic expansion of the distribution of the gradient test statistic is derived for a composite hypothesis under a sequence of Pitman alternative hypotheses converging to the null hypothesis at rate n^{-1/2}, n being the sample size. Comparisons of the local powers of the gradient, likelihood ratio, Wald and score tests reveal no uniform superiority property.

自Nelson和Plosser (1982)後，研究經濟資料是否具有單位根現象，已成為近二十年來熱門且重要的課題。因 為資料性質的不同(恆定或非恆定)，對實證計量模型的設定、統計推論以及原理論的發展有深遠的影響。與傳 統探討單一時間數列之單位根的論文不同的是，本篇論文將橫斷面的資料擴大，探討混合型資料的單位根現象 ( Panel Unit Root )。就此課題，文獻上已有兩個不同的檢定方法: Levin、Lin和Chu (1997)的LLC檢定法以及Im、 Pesaran和Shin (1995)的IPS檢定法。 我們的研究，有別於以上兩者，是從「概似比」的角度(likelihood ratio) 和應用檢定共積關係的Johansen (1988)「Trace檢定」，建構新的單位根檢定統計量。首先於文中推導出，「Trace檢定」可用於檢測單一時間數 列的單位根現象。進而，再將橫斷面資料擴大，採用mean group方法，加總平均每個橫斷面時間數列的「Trace 檢定」統計量，形成混合型資料之單位根檢定統計量 。根據中央極限定理，標準化後的 檢定統計量，極限上 收斂至標準常態分配。此外，我們也推導得出 檢定統計量與傳統ADF、LLC以及IPS檢定統計量極限上的關係。 最後，我們以「蒙地卡羅」模擬方法，分析小樣本下「型一誤差」與「檢定力」的表現。發現新的混合型資 料之單位根檢定統計量表現優良，近似於標準常態分配。故在做混合型資料的單位根分析時，採用 檢定統計 量，可得到較精確的推論。

本文使用Esscher轉換法推導狀態轉換模型、跳躍獨立風險下狀狀態轉換模型及跳躍相關風險下狀態轉換模型的選擇權定價公式。藉由1999年至2011年道瓊工業指數真實市場資料使用EM演算法估計模型參數並使用概似比檢定得到跳躍相關風險下狀態轉換模型最適合描述報酬率資料。接著進行敏感度分析得知，高波動狀態的機率、報酬率的整體波動度及跳躍頻率三者與買權呈現正相關。最後由市場驗證可知，跳躍相關風險下狀態轉換模型在價平及價外的定價誤差皆是最小，在價平的定價誤差則略高於跳躍獨立風險下狀態轉換模型。 / In this paper, we derive regime switching model, regime switching model with independent jump and regime switching model with dependent jump by Esscher transformation. We use the data from 1999 to 2011 Dow-Jones industrial average index market price to estimate the parameter by EM algorithm. Then we use likelihood ratio test to obtain that regime switching model with dependent jump is the best model to depict return data. Moreover, we do sensitivity analysis and find the result that the probability of the higher volatility state , the overall volatility of rate of return , and the jump frequency are positively correlated with call option value. Finally, we enhance the empirical value of regime switching model with dependent jump by means of calculating the price error.

L’objectif de ces travaux de thèse est de développer des architectures fiables de surveillance et de détection des défauts d’une machine asynchrone basées sur des techniques paramétriques de traitement du signal. Pour analyser et détecter les défauts, un modèle paramétrique du courant statorique en environnement stationnaire est proposé. Il est supposé être constitué de plusieurs sinusoïdes avec des paramètres inconnus dans le bruit. Les paramètres de ce modèle sont estimés à l’aide des techniques paramétriques telles que les estimateurs spectraux de type sous-espaces (MUSIC et ESPRIT) et l’estimateur du maximum de vraisemblance. Un critère de sévérité des défauts, basé sur l’estimation des amplitudes des composantes fréquentielles du courant statorique, est aussi proposé pour évaluer le niveau de défaillance de la machine. Un nouveau détecteur des défauts est aussi proposé en utilisant la théorie de détection. Il est principalement basé sur le test du rapport de vraisemblance généralisé avec un signal et un bruit à paramètres inconnus. Enfin, les techniques paramétriques proposées ont été évaluées à l’aide de signaux de courant statoriques expérimentaux de machines asynchrones en considérant les défauts de roulements et les ruptures de barres rotoriques. L’analyse des résultats expérimentaux montre clairement l’efficacité et la capacité de détection des techniques paramétriques proposées. / This Ph.D. thesis aims to develop reliable and cost-effective condition monitoring and faults detection architectures for induction machines. These architectures are mainly based on advanced parametric signal processing techniques. To analyze and detect faults, a parametric stator current model under stationary conditions has been considered. It is assumed to be multiple sinusoids with unknown parameters in noise. This model has been estimated using parametric techniques such as subspace spectral estimators and maximum likelihood estimator. A fault severity criterion based on the estimation of the stator current frequency component amplitudes has also been proposed to determine the induction machine failure level. A novel faults detector based on hypothesis testing has been also proposed. This detector is mainly based on the generalized likelihood ratio test detector with unknown signal and noise parameters. The proposed parametric techniques have been evaluated using experimental stator current signals issued from induction machines under two considered faults: bearing and broken rotor bars faults.Experimental results show the effectiveness and the detection ability of the proposed parametric techniques.

Submitted by Fabio Sobreira Campos da Costa (fabio.sobreira@ufpe.br) on 2017-04-25T14:46:06Z No. of bitstreams: 2 license_rdf: 1232 bytes, checksum: 66e71c371cc565284e70f40736c94386 (MD5) TESE VERSÃO FINAL (CD).pdf: 688894 bytes, checksum: fc5c0291423dc50d4989c1c2d8d4af65 (MD5) / Made available in DSpace on 2017-04-25T14:46:06Z (GMT). No. of bitstreams: 2 license_rdf: 1232 bytes, checksum: 66e71c371cc565284e70f40736c94386 (MD5) TESE VERSÃO FINAL (CD).pdf: 688894 bytes, checksum: fc5c0291423dc50d4989c1c2d8d4af65 (MD5) Previous issue date: 2016-11-04 / Young e Bakir (1987) propôs a classe de Modelos Lineares Log-Gama Generalizados (MLLGG) para analisar dados de sobrevivência. No nosso trabalho, estendemos a classe de modelos propostapor Young e Bakir (1987) permitindo uma estrutura não linear para os parâmetros de regressão. A nova classe de modelos é denominada como Modelos Não Lineares Log-Gama Generalizados (MNLLGG). Com o objetivo de obter a correção de viés de segunda ordem dos estimadores de máxima verossimilhança (EMV) na classe dos MNLLGG, desenvolvemos uma expressão matricial fechada para o estimador de viés de Cox e Snell (1968). Analisamos, via simulação de Monte Carlo, os desempenhos dos EMV e suas versões corrigidas via Cox e Snell (1968) e através da metodologia bootstrap (Efron, 1979). Propomos também resíduos e técnicas de diagnóstico para os MNLLGG, tais como: alavancagem generalizada, influência local e influência global. Obtivemos, em forma matricial, uma expressão para o fator de correção de Bartlett à estatística da razão de verossimilhanças nesta classe de modelos e desenvolvemos estudos de simulação para avaliar e comparar numericamente o desempenho dos testes da razão de verossimilhanças e suas versões corrigidas em relação ao tamanho e poder em amostras finitas. Além disso, derivamos expressões matriciais para os fatores de correção tipo-Bartlett às estatísticas escore e gradiente. Estudos de simulação foram feitos para avaliar o desempenho dos testes escore, gradiente e suas versões corrigidas no que tange ao tamanho e poder em amostras finitas. / Young e Bakir (1987) proposed the class of generalized log-gamma linear regression models (GLGLM) to analyze survival data. In our work, we extended the class of models proposed by Young e Bakir (1987) considering a nonlinear structure for the regression parameters. The new class of models is called generalized log-gamma nonlinear regression models (GLGNLM). We also propose matrix formula for the second-order bias of the maximum likelihood estimate of the regression parameter vector in the GLGNLM class. We use the results by Cox and Snell (1968) and bootstrap technique [Efron (1979)] to obtain the bias-corrected maximum likelihood estimate. Residuals and diagnostic techniques were proposed for the GLGNLM, such as generalized leverage, local and global influence. An general matrix notation was obtained for the Bartlett correction factor to the likelihood ratio statistic in this class of models. Simulation studies were developed to evaluate and compare numerically the performance of likelihood ratio tests and their corrected versions regarding size and power in finite samples. Furthermore, general matrix expressions were obtained for the Bartlett-type correction factor for the score and gradient statistics. Simulation studies were conducted to evaluate the performance of the score and gradient tests with their corrected versions regarding to the size and power in finite samples.

Rieck & Nedelman (1991) propuseram um modelo de regressão log-linear tendo como base a distribuição Birnbaum-Saunders (Birnbaum & Saunders, 1969a). O modelo proposto pelos autores vem sendo bastante explorado e tem se mostrado uma ótima alternativa a outros modelos propostos na literatura, como por exemplo, os modelos de regressão Weibull, gama e lognormal. No entanto, até o presente momento, não existe nenhum estudo tratando de refinamentos para as estatísticas da razão de verossimilhanças e escore nesta classe de modelos de regressão. Assim, um dos objetivos desta tese é obter um fator de correção de Bartlett para a estatística da razão de verossimilhanças e um fator de correção tipo-Bartlett para a estatística escore nesse modelo. Estes ajustes melhoram a aproximação da distribuição nula destas estatísticas pela distribuição qui-quadrado de referência. Adicionalmente, objetiva-se obter ajustes para a estatística da razão de verossimilhanças sinalizada. Tais ajustes melhoram a aproximação desta estatística pela distribuição normal padrão. Recentemente, uma nova estatística de teste foi proposta por Terrell (2002), a qual o autor denomina estatística gradiente. Esta estatística foi derivada a partir da estatística escore e da estatística de Wald modificada (Hayakawa & Puri, 1985). A combinação daquelas duas estatísticas resulta em uma estatística muito simples de ser calculada, não envolvendo, por exemplo, nenhum cálculo matricial como produto e inversa de matrizes. Esta estatística foi recentemente citada por Rao (2005): \"The suggestion by Terrell is attractive as it is simple to compute. It would be of interest to investigate the performance of the [gradient] statistic.\" Caminhando na direção da sugestão de Rao, outro objetivo da tese é obter uma expansão assintótica para a distribuição da estatística gradiente sob uma sequência de alternativas de Pitman convergindo para a hipótese nula a uma taxa de convergência de n^{-1/2} utilizando a metodologia desenvolvida por Peers (1971) e Hayakawa (1975). Em particular, mostramos que, até ordem n^{-1/2}, a estatística gradiente segue distribuição qui-quadrado central sob a hipótese nula e distribuição qui-quadrado não central sob a hipótese alternativa. Também temos como objetivo comparar o poder local deste teste com o poder local dos testes da razão de verossimilhanças, de Wald e escore. Finalmente, aplicaremos a expansão assintótica derivada na tese em algumas classes particulares de modelos. / The Birnbaum-Saunders regression model is commonly used in reliability studies.We address the issue of performing inference in this class of models when the number of observations is small. Our simulation results suggest that the likelihood ratio and score tests tend to be liberal when the sample size is small. We derive Bartlett and Bartlett-type correction factors which reduce the size distortion of the tests. Additionally, we also consider modified signed log-likelihood ratio statistics in this class of models. Finally, the asymptotic expansion of the distribution of the gradient test statistic is derived for a composite hypothesis under a sequence of Pitman alternative hypotheses converging to the null hypothesis at rate n^{-1/2}, n being the sample size. Comparisons of the local powers of the gradient, likelihood ratio, Wald and score tests reveal no uniform superiority property.

This paper deals with sequential likelihood ratio tests based on grouped observations. It is demonstrated that the method of conjugated parameter pairs known from the non-grouped case can be extended to the grouped case obtaining Waldlike approximations for the OC- and ASN- function. For near hypotheses so-called F-optimal groupings are recommended. As example an SPRT based on grouped observations for the parameter of an exponentially distributed random variable is considered.

Survival analytic methods help to analyze lifetime data arising from medical and reliability experiments. The popular proportional hazards model, proposed by Cox (1972), is widely used in survival analysis to study the effect of risk factors on lifetimes. An important assumption in regression type analysis is that all relative risk factors should be included in the model. However, not all relative risk factors are observed due to measurement difficulty, inaccessibility, cost considerations, and so on. These unobservable risk factors can be modelled by the so-called frailty model, originally introduced by Vaupel et al. (1979). Furthermore, the frailty model is also applicable to clustered data. Cluster data possesses the feature that observations within the same cluster share similar conditions and environment, which are sometimes difficult to observe. For example, patients from the same family share similar genetics, and patients treated in the same hospital share the same group of profes- sionals and same environmental conditions. These factors are indeed hard to quantify or measure. In addition, this type of similarity introduces correlation among subjects within clusters. In this thesis, a semi-parametric frailty model is proposed to address aforementioned issues. The baseline hazards function is approximated by a piecewise constant function and the frailty distribution is assumed to be a Birnbaum-Saunders distribution. Due to the advancement in modern medical sciences, many diseases are curable, which in turn leads to the need of incorporating cure proportion in the survival model. The frailty model discussed here is further extended to a mixture cure rate frailty model by integrating the frailty model into the mixture cure rate model proposed originally by Boag (1949) and Berkson and Gage (1952). By linking the covariates to the cure proportion through logistic/logit link function and linking observable covariates and unobservable covariates to the lifetime of the uncured population through the frailty model, we obtain a flexible model to study the effect of risk factors on lifetimes. The mixture cure frailty model can be reduced to a mixture cure model if the effect of frailty term is negligible (i.e., the variance of the frailty distribution is close to 0). On the other hand, it also reduces to the usual frailty model if the cure proportion is 0. Therefore, we can use a likelihood ratio test to test whether the reduced model is adequate to model the given data. We assume the baseline hazard to be that of Weibull distribution since Weibull distribution possesses increasing, constant or decreasing hazard rate, and the frailty distribution to be Birnbaum-Saunders distribution. D ́ıaz-Garc ́ıa and Leiva-Sa ́nchez (2005) proposed a new family of life distributions, called generalized Birnbaum-Saunders distribution, including Birnbaum-Saunders distribution as a special case. It allows for various degrees of kurtosis and skewness, and also permits unimodality as well as bimodality. Therefore, integration of a generalized Birnbaum-Saunders distribution as the frailty distribution in the mixture cure frailty model results in a very flexible model. For this general model, parameter estimation is carried out using a marginal likelihood approach. One of the difficulties in the parameter estimation is that the likelihood function is intractable. The current technology in computation enables us to develop a numerical method through Monte Carlo simulation, and in this approach, the likelihood function is approximated by the Monte Carlo method and the maximum likelihood estimates and standard errors of the model parameters are then obtained numerically by maximizing this approximate likelihood function. An EM algorithm is also developed for the Birnbaum-Saunders mixture cure frailty model. The performance of this estimation method is then assessed by simulation studies for each proposed model. Model discriminations is also performed between the Birnbaum-Saunders frailty model and the generalized Birnbaum-Saunders mixture cure frailty model. Some illustrative real life examples are presented to illustrate the models and inferential methods developed here. / Thesis / Doctor of Science (PhD)

Cure rate models, introduced by Boag (1949), are very commonly used while modelling lifetime data involving long time survivors. Applications of cure rate models can be seen in biomedical science, industrial reliability, finance, manufacturing, demography and criminology. In this thesis, cure rate models are discussed under a competing cause scenario, with the assumption of proportional odds (PO) lifetime distributions for the susceptibles, and statistical inferential methods are then developed based on right-censored data. In Chapter 2, a flexible cure rate model is discussed by assuming the number of competing causes for the event of interest following the Conway-Maxwell (COM) Poisson distribution, and their corresponding lifetimes of non-cured or susceptible individuals can be described by PO model. This provides a natural extension of the work of Gu et al. (2011) who had considered a geometric number of competing causes. Under right censoring, maximum likelihood estimators (MLEs) are obtained by the use of expectation-maximization (EM) algorithm. An extensive Monte Carlo simulation study is carried out for various scenarios, and model discrimination between some well-known cure models like geometric, Poisson and Bernoulli is also examined. The goodness-of-fit and model diagnostics of the model are also discussed. A cutaneous melanoma dataset example is used to illustrate the models as well as the inferential methods. Next, in Chapter 3, the destructive cure rate models, introduced by Rodrigues et al. (2011), are discussed under the PO assumption. Here, the initial number of competing causes is modelled by a weighted Poisson distribution with special focus on exponentially weighted Poisson, length-biased Poisson and negative binomial distributions. Then, a damage distribution is introduced for the number of initial causes which do not get destroyed. An EM-type algorithm for computing the MLEs is developed. An extensive simulation study is carried out for various scenarios, and model discrimination between the three weighted Poisson distributions is also examined. All the models and methods of estimation are evaluated through a simulation study. A cutaneous melanoma dataset example is used to illustrate the models as well as the inferential methods. In Chapter 4, frailty cure rate models are discussed under a gamma frailty wherein the initial number of competing causes is described by a Conway-Maxwell (COM) Poisson distribution in which the lifetimes of non-cured individuals can be described by PO model. The detailed steps of the EM algorithm are then developed for this model and an extensive simulation study is carried out to evaluate the performance of the proposed model and the estimation method. A cutaneous melanoma dataset as well as a simulated data are used for illustrative purposes. Finally, Chapter 5 outlines the work carried out in the thesis and also suggests some problems of further research interest. / Thesis / Doctor of Philosophy (PhD)