yes / This paper presents a methodology for fault detection, fault prediction and fault isolation based on the integration of hidden Markov modelling (HMM) and Bayesian networks (BN). This addresses the nonlinear and non-Gaussian data characteristics to support fault detection and prediction, within an explainable hybrid framework that captures causality in the complex engineered system. The proposed methodology is based on the analysis of the pattern of similarity in the log-likelihood (LL) sequences against the training data for the mixture of Gaussians HMM (MoG-HMM). The BN model identifies the root cause of detected/predicted faults, using the information propagated from the HMM model as empirical evidence. The feasibility and effectiveness of the presented approach are discussed in conjunction with the application to a real-world case study of an automotive exhaust gas Aftertreatment system. The paper details the implementation of the methodology to this case study, with data available from real-world usage of the system. The results show that the proposed methodology identifies the fault faster and attributes the fault to the correct root cause. While the proposed methodology is illustrated with an automotive case study, its applicability is much wider to the fault detection and prediction problem of any similar complex engineered system.

Patents are a fundamental part of scientific and engineering work, ensuringprotection of inventions owned by individuals or organizations. Patents areusually made public 18 months after being filed to a patent office, whichmeans that current publicly available patent data only provides informationabout the past. Regression models applied on discrete time series can be usedas a prediction tool to counteract this, building a 18 month long bridge intothe future and beyond. While linear models are popular for their simplicity,Bayesian networks have statistical properties that can produce high forecastingquality. Improvements is also made by using signal processing as patentdata is naturally stochastic. This thesis implements wavelet-based signalprocessing and P CA to increase stability and reduce overfitting. A multiplelinear regression model and a Bayesian network model is then designed andapplied to the transformed data. When evaluated on each data set, the Bayesianmodel both performs better and exhibits greater stability and consistency inits predictions. As expected, the linear model is both smaller and faster toevaluate and train. Despite an increase in complexity and slower evaluationtimes, the Bayesian model is conclusively superior to the linear model. Futurework should focus on the signal processing method and additional layers inthe Bayesian network. / Patent är en grundläggande byggsten av den tekniska världen då de skyddaruppfinningar som ägs av individer eller organisationer. Patent publicerasvanligtvis 18 månader efter att de lämnats in till ett patentverk, vilket innebäratt patentdata som är tillgänglig idag endast ger information om det förflutna.Regressionsmodeller som förutspår diskreta tidsserier kan användas somett verktyg för att motverka detta. Då linjära modeller är populära för sinenkelhet, har Bayesianska nätverk statistiska egenskaper som kan produceramodeller med hög kvalité. Patentdata är naturligt kaotisk och måste bearbetasinnan en modell använder den. Denna uppsats implementerar wavelet-baseradsignalbehandling och P CA som förbättrar stabilitet och kvalité. En linjärregressionsmodell och en Bayesiansk nätverksmodell designas och applicerassedan på transformerad data. I varje enskilt fall presterar den Bayesianskamodellen bättre med stabila och konsekventa förutsägelser. Som förväntatär den linjära modellen snabbare att både använda och träna. Trots en ökadkomplexitet och långsammare evaluering är den Bayesianska modellen ettsjälvklart val över den linjära modellen. Framtida förbättringar bör fokuserapå behandling av indata och komplexiteten i det Bayesianska nätverket.

The complexity of engineered systems has increased remarkably to meet customer needs. In the continuously growing global market, it is essential for engineered systems to keep their productivities which can be achieved by higher reliability and availability. Integrated health management based on diagnostics and prognostics provides significant benefits, which includes increasing system safety and operational reliability, with a significant impact on the life-cycle costs, reducing operating costs and increasing revenues. Characteristics of complex systems such as nonlinearity, dynamicity, non-stationarity, and non-Gaussianity make diagnostics and prognostics more challenging tasks and decrease the application of classic reliability methods remarkably – as they cannot address the dynamic behaviour of these systems. This research has focused on detecting, predicting and isolating faults in engineered systems, using operational data with multifarious data characteristics. Complexities in the data, including non-Gaussianity and high nonlinearity, impose stringent challenges on fault analysis. To deal with these challenges, this research proposed an integrated data-driven methodology in which hidden Markov modelling (HMM) and Bayesian network (BN) were employed to detect, predict and isolate faults in a system. The fault detection and prediction were based on comparing and exploiting pattern similarity in the data via the loglikelihood values generated through HMM training. To identify the root cause of the faults, the probability values obtained from updating the BN were used which were based on the virtual evidence provided by HMM training and log-likelihood values. To set up a more accurate data-driven model – particularly BN structure – engineering analyses were employed in a structured way to explore the causal relationships in the system which is essential for reliability analysis of complex engineered systems. The automotive exhaust gas Aftertreatment system is a complex engineered system consisting of several subsystems working interdependently to meet emission legislations. The Aftertreatment system is a highly nonlinear, dynamic and non-stationary system. Consequently, it has multifarious data characteristics, where these characteristics raise the challenges of diagnostics and prognostics for this system, compared to some of the references systems, such as the Tennessee Eastman process or rolling bearings. The feasibility and effectiveness of the presented framework were discussed in conjunction with the application to a real-world case study of an exhaust gas Aftertreatment system which provided good validation of the methodology, proving feasibility to detect, predict, and isolate unidentified faults in dynamic processes.

Bayesian network (BN) structure learning from data has been an active research area in the machine learning field in recent decades. Much of the research has considered BN structure learning as an optimization problem. However, the finding of optimal BN from data is NP-hard. This fact has driven the use of heuristic algorithms for solving this kind of problem. Amajor recent focus in BN structure learning is on search and score algorithms. In these algorithms, a scoring function is introduced and a heuristic search algorithm is used to evaluate each network with respect to the training data. The optimal network is produced according to the best score evaluated. This thesis investigates a range of search and score algorithms to understand the relationship between technique performance and structure features of the problems. The main contributions of this thesis include (a) Two novel Ant Colony Optimization based search and score algorithms for BN structure learning; (b) Node juxtaposition distribution for studying the relationship between the best node ordering and the optimal BN structure; (c) Fitness landscape analysis for investigating the di erent performances of both chain score function and the CH score function; (d) A classifier method is constructed by utilizing receiver operating characteristic curve with the results on fitness landscape analysis; and finally (e) a selective o -line hyperheuristic algorithm is built for unseen BN structure learning with search and score algorithms. In this thesis, we also construct a new algorithm for producing BN benchmark structures and apply our novel approaches to a range of benchmark problems and real world problem.

Approved for public release; distribution in unlimited. / The principles of Sun Tzu's Art of War have been widely used by business executives and military officers with much success in the realm of competition and conflict. However, when conflict situations arise in a highly stressful environment coupled with the pressure of time, decision makers may not be able to consider all the key concepts when forming their decisions or strategies. Therefore, a structured reasoning approach may be used to apply Sun Tzu's principles correctly and fully. Sun Tzu's principles are believed to be able to be modeled mathematically; hence, a Bayesian Network model (a form of mathematical tool using probability theory) is used to capture Sun Tzu's principles and provide the structured reasoning approach. Scholars have identified incompleteness in Sun Tzu's appreciation of information in war and his application of secret agents. This incompleteness resulted in circular reasoning when both sides of the conflict apply his principles. This circular reasoning can be resolved through the use of advanced probability theory. A Bayesian Network Model however, not only provides a structured reasoning approach, but more importantly, it can also resolve the circular reasoning problem that has been identified. / Captain, Singapore Army

This study proposes an accuracy comparison of two of the best performing machine learning algorithms in natural language processing, the Bayesian Network and the Long Short-Term Memory (LSTM) Recurrent Neural Network, in detecting Alzheimer’s disease symptoms in conversation transcripts. Because of the current global rise of life expectancy, the number of seniors affected by Alzheimer’s disease worldwide is increasing each year. Early detection is important to ensure that affected seniors take measures to relieve symptoms when possible or prepare plans before further cognitive decline occurs. Literature shows that natural language processing can be a valid tool for early diagnosis of the disease. This study found that mild dementia and possible Alzheimer’s can be detected in conversation transcripts with promising results, and that the LSTM is particularly accurate in said detection, reaching an accuracy of 86.5% on the chosen dataset. The Bayesian Network classified with an accuracy of 72.1%. The study confirms the effectiveness of a natural language processing approach to detecting Alzheimer’s disease.

Dans cette thèse, nous nous intéressons au problème spécifique de l'apprentissage de structure de modèles graphiques probabilistes, c'est-à-dire trouver la structure la plus efficace pour représenter une distribution, à partir seulement d'un ensemble d'échantillons D ∼ p(v). Dans une première partie, nous passons en revue les principaux modèles graphiques probabilistes de la littérature, des plus classiques (modèles dirigés, non-dirigés) aux plus avancés (modèles mixtes, cycliques etc.). Puis nous étudions particulièrement le problème d'apprentissage de structure de modèles dirigés (réseaux Bayésiens), et proposons une nouvelle méthode hybride pour l'apprentissage de structure, H2PC (Hybrid Hybrid Parents and Children), mêlant une approche à base de contraintes (tests statistiques d'indépendance) et une approche à base de score (probabilité postérieure de la structure). Dans un second temps, nous étudions le problème de la classification multi-label, visant à prédire un ensemble de catégories (vecteur binaire y P (0, 1)m) pour un objet (vecteur x P Rd). Dans ce contexte, l'utilisation de modèles graphiques probabilistes pour représenter la distribution conditionnelle des catégories prend tout son sens, particulièrement dans le but minimiser une fonction coût complexe. Nous passons en revue les principales approches utilisant un modèle graphique probabiliste pour la classification multi-label (Probabilistic Classifier Chain, Conditional Dependency Network, Bayesian Network Classifier, Conditional Random Field, Sum-Product Network), puis nous proposons une approche générique visant à identifier une factorisation de p(y|x) en distributions marginales disjointes, en s'inspirant des méthodes d'apprentissage de structure à base de contraintes. Nous démontrons plusieurs résultats théoriques, notamment l'unicité d'une décomposition minimale, ainsi que trois procédures quadratiques sous diverses hypothèses à propos de la distribution jointe p(x, y). Enfin, nous mettons en pratique ces résultats afin d'améliorer la classification multi-label avec les fonctions coût F-loss et zero-one loss / In this thesis, we address the specific problem of probabilistic graphical model structure learning, that is, finding the most efficient structure to represent a probability distribution, given only a sample set D ∼ p(v). In the first part, we review the main families of probabilistic graphical models from the literature, from the most common (directed, undirected) to the most advanced ones (chained, mixed etc.). Then we study particularly the problem of learning the structure of directed graphs (Bayesian networks), and we propose a new hybrid structure learning method, H2PC (Hybrid Hybrid Parents and Children), which combines a constraint-based approach (statistical independence tests) with a score-based approach (posterior probability of the structure). In the second part, we address the multi-label classification problem, which aims at assigning a set of categories (binary vector y P (0, 1)m) to a given object (vector x P Rd). In this context, probabilistic graphical models provide convenient means of encoding p(y|x), particularly for the purpose of minimizing general loss functions. We review the main approaches based on PGMs for multi-label classification (Probabilistic Classifier Chain, Conditional Dependency Network, Bayesian Network Classifier, Conditional Random Field, Sum-Product Network), and propose a generic approach inspired from constraint-based structure learning methods to identify the unique partition of the label set into irreducible label factors (ILFs), that is, the irreducible factorization of p(y|x) into disjoint marginal distributions. We establish several theoretical results to characterize the ILFs based on the compositional graphoid axioms, and obtain three generic procedures under various assumptions about the conditional independence properties of the joint distribution p(x, y). Our conclusions are supported by carefully designed multi-label classification experiments, under the F-loss and the zero-one loss functions

Le projet Européen « GENIUS » ambitionne de développer les méthodologies génériques pour le diagnostic de systèmes piles à combustible à haute température de type oxyde solide (SOFC). Le travail de cette thèse s’intègre dans ce projet ; il a pour objectif la mise en oeuvre d’un outil de diagnostic en utilisant le stack comme capteur spécial pour détecter et identifierles défaillances dans les sous-systèmes du stack SOFC.Trois algorithmes de diagnostic ont été développés, se basant respectivement sur la méthode de classification k-means, la technique de décomposition du signal en ondelettes ainsi que la modélisation par réseau Bayésien. Le premier algorithme sert au diagnostic ex-situ et est appliqué pour traiter les donnés issues des essais de polarisation. Il permet de déterminer les variables de réponse significatives qui indiquent l’état de santé du stack. L’indice Silhouette a été calculé comme mesure de qualité de classification afin de trouver le nombre optimal de classes dans la base de données.La détection de défaut en temps réel peut se réaliser par le deuxième algorithme. Puisque le stack est employé en tant que capteur, son état de santé doit être vérifié préalablement. La transformée des ondelettes a été utilisée pour décomposer les signaux de tension de la pile SOFC dans le but de chercher les variables caractéristiques permettant d’indiquer l’état desanté de la pile et également assez discriminatives pour différentier les conditions d’opération normales et anormales.Afin d’identifier le défaut du système lorsqu’une condition d’opération anormale s’est détectée, les paramètres opérationnelles réelles du stack doivent être estimés. Un réseau Bayésien a donc été développé pour accomplir ce travail.Enfin, tous les algorithmes ont été validés avec les bases de données expérimentales provenant de systèmes SOFC variés, afin de tester leur généricité. / The EU-project “GENIUS” is targeted at the investigation of generic diagnosis methodologies for different Solid Oxide Fuel Cell (SOFC) systems. The Ph.D study presented in this thesis was integrated into this project; it aims to develop a diagnostic tool for SOFC system fault detection and identification based on validated diagnostic algorithms, through applying theSOFC stack as a sensor.In this context, three algorithms, based on the k-means clustering technique, the wavelet transform and the Bayesian method, respectively, have been developed. The first algorithm serves for ex-situ diagnosis. It works on the classification of the polarization measurements of the stack, aiming to figure out the significant response variables that are able to indicate the state of health of the stack. The parameter “Silhouette” has been used to evaluate the classification solutions in order to determine the optimal number of classes/patterns to retain from the studied database.The second algorithm allows the on-line fault detection. The wavelet transform has been used to decompose the SOFC’s voltage signals for the purpose of finding out the effective feature variables that are discriminative for distinguishing the normal and abnormal operating conditions of the system. Considering the SOFC as a sensor, its reliability must be verifiedbeforehand. Thus, the feature variables are also required to be indicative to the state of health of the stack.When the stack is found being operated improperly, the actual operating parameters should be estimated so as to identify the system fault. To achieve this goal, a Bayesian network has been proposed serving as a meta-model of the stack to accomplish the estimation. At the end, the databases originated from different SOFC systems have been used to validate these three algorithms and assess their generalizability.

Sistemas Instrumentados de Segurança (SIS) são projetados para prevenir e/ou mitigar acidentes, evitando indesejáveis cenários com alto potencial de risco, assegurando a proteção da saúde das pessoas, proteção do meio ambiente e economia de custos com equipamentos industriais. Desta forma, é extremamente recomendado neste projeto de SIS o uso de métodos formais para garantir as especificações de segurança em conformidade com as normas regulamentadoras vigentes, principalmente para atingir o nível de integridade de segurança (SIL) desejado. Adicionalmente, algumas das normas de segurança como ANSI / ISA S.84.01; IEC 61508, IEC 61511, entre outras, recomendam uma série de procedimentos relacionados ao ciclo de vida de segurança de um projeto de SIS. Desta forma, destacam-se as atividades que compreendem o desenvolvimento e a validação dos algoritmos de controle em que se separam semanticamente os aspectos voltados para o diagnóstico de falhas críticas e o tratamento destas falhas associado a um controle de coordenação para filtrar a ocorrência de falhas espúrias. Portanto, a contribuição deste trabalho é propor um método formal para a modelagem e análise de SIS, incluindo o diagnóstico e o tratamento de falhas críticas, baseado em rede Bayesiana (BN) e rede de Petri (PN). Este trabalho considera o diagnóstico e o tratamento para cada função instrumentada de segurança (SIF) a partir do resultado do estudo de análise de riscos, de acordo com a metodologia de HAZOP (Hazard and Operability). / Safety Instrumented Systems (SIS) are design to prevent and/or mitigate accidents, avoiding undesirable high potential risk scenarios, assuring protection of people health, protecting the environment and saving costs of industrial equipment. It is strongly recommended in this design formal method to assure the safety specifications in accordance to standards regulations, mainly for reaching desired safety integrity level (SIL). Additionally, some of the safety standards such as ANSI/ISA S.84.01; IEC 61508, IEC 61511, among others, guide different activities related to Safety Life Cycle (SLC) design of SIS. In special, there are design activities that involves the development and validation of control algorithm that separate semantically aspects oriented to diagnosis and treatment of critical faults associated with a control coordination to filter spurious failures occurrence. In this context, the contribution of this work is to propose a formal method for modeling and analysis of SIS designed including diagnostic and treatment of critical faults based on Bayesian networks (BN) and Petri nets (PN). This approach considers diagnostic and treatment for each safety instrumented function (SIF) obtained according hazard and operability (HAZOP) methodology.

Sistemas de suporte a decisão são utilizados em cenários com incertezas. Uma decisão normalmente é auxiliada por resultados obtidos com ações passadas em problemas semelhantes. Quando um sistema de suporte a decisão incorpora conhecimento específico de uma área, estes recebem o nome de sistemas especialistas. Tal conhecimento especifico é utilizado para inferência juntamente com as informações de entrada a respeito do problema. O objetivo deste trabalho é a avaliação e modelagem de sistemas de auxílio a decisão, foram analisadas duas abordagens para um mesmo problema alvo, sendo uma de gerenciamento do problema e outra de detecção do problema. A abordagem de gerenciamento utiliza redes Bayesianas para modelagem, tanto do conhecimento específico quanto para a inferência. As variáveis utilizadas, as relações de dependência e as probabilidades condicionais entre as variáveis foram extraídas da literatura. A abordagem de detecção do problema utilizou imagens para extração de características seguida de um algoritmo de agrupamento para comparação com a classificação de um especialista. Uma das áreas de aplicação de sistemas especialistas é na área clínica, podendo auxiliar tanto na detecção, diagnóstico e tratamento de doenças. A cárie dental é um problema generalizado que afeta a maioria das pessoas, tanto em países ricos, como em países pobres. Existem poucos sistemas para auxílio no processo de diagnóstico da cárie, sendo a maior parte dos sistemas existentes determinísticos, focando apenas na detecção da lesão. O sistema de gerenciamento da cárie desenvolvido foi apresentado a dois profissionais da odontologia, a opinião deles mostra que está abordagem é promissora e aplicável em campos como a educação e a atenção básica a saúde. Além da apresentação aos profissionais, foram utilizados casos bem estabelecidos da literatura para analisar as sugestões fornecidas pela Rede, e o resultado foi coerente com o cenário real de tomada de decisão. A metodologia de detecção da cárie resultou em um alto valor de acurácia, 96.88%, mostrando que tal metodologia é promissora em comparação com outros trabalhos da área. Além da contribuição para a área de informática odontológica, os resultados mostram que a extração da estrutura e das probabilidades condicionais da rede a partir da literatura é uma metodologia que pode ser utilizada em outras áreas com cenário similar ao do diagnóstico da cárie. Nos próximos passos do projeto alguns pontos referentes a modelagem de sistemas e redes Bayesianas serão analisados, como escalabilidade e testes de validação, tanto quantitativamente como qualitativamente, isto inclui o desenvolvimento de métodos computacionalmente efetivos para a geração de casos aleatórios utilizando o Método de Monte Carlo / Decision support systems are used in uncertainty scenarios; normally a decision is choose using similar problems actions results. Decision support systems could incorporate specific knowledge; such systems are called expert systems. The specific knowledge is used for inference about the problem scenario. This work objective is the evaluation and modeling of decision support systems, we analyzed two distinct approaches for the same problem, one for detection, another for management. The management approach uses Bayesian networks for modeling the specific knowledge and the inference engine. The variables choice, the dependences relationship and the conditional probabilities were extracted from the scientific literature. The detection approach used images and feature extraction to perform a clustering and compare the output labels with a specialist classification. One application of expert systems is clinical, supporting diseases detection, diagnosis and treatment. Dental caries is a generalized problem that affects major part of the population, few systems exists for support the caries diagnostic process, the major part is deterministic, focusing only the detection problem. The caries management system developed here was shown to two odontology professionals, and they opinion encourage such approach to be applied in fields like odontology education and basic health. Beyond this, we used well-established cases to analyze the network output suggestions, the result obtained was coherent with the real decision making scenario. The caries detection approach resulted in a high accuracy, 96.88%, showing that methodology is promising. Besides the contribution for dental informatics field, the results obtained here shows that the extraction of the network structure from the literature could be used in problems similar with caries diagnoses. The project next steps are to analyze some points of systems modeling and Bayesian networks, like scalability and validation tests, both quantitative and qualitative, and including the development of computational effectives methods for the use of Monte Carlo methodology