<p>One in nine women is expected to be diagnosed with breast cancer during her life. In 2013, an estimated 23, 800 Canadian women will be diagnosed with breast cancer and 5, 000 will die of it. Making decisions about the treatment for a patient is difficult since it depends on various clinical features, genomic factors, and pathological and cellular classification of a tumor.</p> <p>In this research, we propose a probabilistic graphical model for prognosis and diagnosis of breast cancer that can help medical doctors make better decisions about the best treatment for a patient. Probabilistic graphical models are suitable for making decisions under uncertainty from big data with missing attributes and noisy evidence.</p> <p>Using the proposed model, we may enter the results of different tests (e.g. estrogen and progesterone receptor test and HER2/neu test), microarray data, and clinical traits (e.g. woman's age, general health, menopausal status, stage of cancer, and size of the tumor) to the model and answer to following questions. How likely is it that the cancer will extend in the body (distant metastasis)? What is the chance of survival? How likely is that the cancer comes back (local or regional recurrence)? How promising is a treatment? For example, how likely metastasis is and how likely recurrence is for a new patient, if certain treatment e.g. surgical removal, radiation therapy, hormone therapy, or chemotherapy is applied. We can also classify various types of breast cancers using this model.</p> <p>Previous work mostly relied on clinical data. In our opinion, since cancer is a genetic disease, the integration of the genomic (microarray) and clinical data can improve the accuracy of the model for prognosis and diagnosis. However, increasing the number of variables may lead to poor results due to the curse of dimensionality dilemma and small sample size problem. The microarray data is high dimensional. It consists of around 25, 000 variables per patient. Moreover, structure learning and parameter learning for probabilistic graphical models require a significant amount of computations. The number of possible structures is also super-exponential with respect to the number of variables. For instance, there are more than 10^18 possible structures with just 10 variables.</p> <p>We address these problems by applying manifold learning and dimensionality reduction techniques to improve the accuracy of the model. Extensive experiments using real-world data sets such as METRIC and NKI show the accuracy of the proposed method for classification and predicting certain events, like recurrence and metastasis.</p> / Master of Science (MSc)

The Maximum Likelihood Degree (ML degree) of a statistical model is the number of complex critical points of the likelihood function. In this thesis we study this on Colored Gaussian Graphical Models, classifying the ML degree of colored graphs of order up to three. We do this by calculating the rational function degree of the gradient of the log- likelihood. Moreover we find that coloring a graph can lower the ML degree. Finally we calculate solutions to the homaloidal partial differential equation developed by Améndola et al. The code developed for these calculations can be used on graphs of higher orders. / Maximum likelihood-graden (ML-graden) för en statistisk modell är antalet komplexa kritiska punkter för likelihoodfunktionen. I denna avhandling studerar vi detta på färgade Gaussiska grafiska modeller och klassificerar ML-graden för färgade grafer av ordning upp till tre. Detta görs genom att beräkna den rationella funktionsgraden för gradienten av logaritmen av likelihoodfunktionen. Dessutom finner vi att ML-graden av en graf kan minskas genom att färgläggas. Slutligen beräknar vi lösningar till den homaloidala partiella differentialekvationen utvecklad av Améndola et al. Den kod som utvecklats för dessa beräkningar kan användas på grafer av högre ordning.

Robust speech recognition in acoustic environments that contain multiple speech sources and/or complex non-stationary noise is a difficult problem, but one of great practical interest. The formalism of probabilistic graphical models constitutes a relatively new and very powerful tool for better understanding and extending existing models, learning, and inference algorithms; and a bedrock for the creative, quasi-systematic development of new ones. In this thesis a collection of new graphical models and inference algorithms for robust speech recognition are presented. The problem of speech separation using multiple microphones is first treated. A family of variational algorithms for tractably combining multiple acoustic models of speech with observed sensor likelihoods is presented. The algorithms recover high quality estimates of the speech sources even when there are more sources than microphones, and have improved upon the state-of-the-art in terms of SNR gain by over 10 dB. Next the problem of background compensation in non-stationary acoustic environments is treated. A new dynamic noise adaptation (DNA) algorithm for robust noise compensation is presented, and shown to outperform several existing state-of-the-art front-end denoising systems on the new DNA + Aurora II and Aurora II-M extensions of the Aurora II task. Finally, the problem of speech recognition in speech using a single microphone is treated. The Iroquois system for multi-talker speech separation and recognition is presented. The system won the 2006 Pascal International Speech Separation Challenge, and amazingly, achieved super-human recognition performance on a majority of test cases in the task. The result marks a significant first in automatic speech recognition, and a milestone in computing.

This thesis presents a class of graphical models for directly representing the joint cumulative distribution function (CDF) of many random variables, called cumulative distribution networks (CDNs). Unlike graphical models for probability density and mass functions, in a CDN, the marginal probabilities for any subset of variables are obtained by computing limits of functions in the model. We will show that the conditional independence properties in a CDN are distinct from the conditional independence properties of directed, undirected and factor graph models, but include the conditional independence properties of bidirected graphical models. As a result, CDNs are a parameterization for bidirected models that allows us to represent complex statistical dependence relationships between observable variables. We will provide a method for constructing a factor graph model with additional latent variables for which graph separation of variables in the corresponding CDN implies conditional independence of the separated variables in both the CDN and in the factor graph with the latent variables marginalized out. This will then allow us to construct multivariate extreme value distributions for which both a CDN and a corresponding factor graph representation exist. In order to perform inference in such graphs, we describe the `derivative-sum-product' (DSP) message-passing algorithm where messages correspond to derivatives of the joint cumulative distribution function. We will then apply CDNs to the problem of learning to rank, or estimating parametric models for ranking, where CDNs provide a natural means with which to model multivariate probabilities over ordinal variables such as pairwise preferences. We will show that many previous probability models for rank data, such as the Bradley-Terry and Plackett-Luce models, can be viewed as particular types of CDN. Applications of CDNs will be described for the problems of ranking players in multiplayer team-based games, document retrieval and discovering regulatory sequences in computational biology using the above methods for inference and estimation of CDNs.

Estimation of Distribution Algorithms (EDAs) belong to the class of population based optimisation algorithms. They are motivated by the idea of discovering and exploiting the interaction between variables in the solution. They estimate a probability distribution from population of solutions, and sample it to generate the next population. Many EDAs use probabilistic graphical modelling techniques for this purpose. In particular, directed graphical models (Bayesian networks) have been widely used in EDA. This thesis proposes an undirected graphical model (Markov Random Field (MRF)) approach to estimate and sample the distribution in EDAs. The interaction between variables in the solution is modelled as an undirected graph and the joint probability of a solution is factorised as a Gibbs distribution. The thesis describes a model of fitness function that approximates the energy in the Gibbs distribution, and shows how this model can be fitted to a population of solutions to estimate the parameters of the MRF. The estimated MRF is then sampled to generate the next population. This approach is applied to estimation of distribution in a general framework of an EDA, called Distribution Estimation using Markov Random Fields (DEUM). The thesis then proposes several variants of DEUM using different sampling techniques and tests their performance on a range of optimisation problems. The results show that, for most of the tested problems, the DEUM algorithms significantly outperform other EDAs, both in terms of number of fitness evaluations and the quality of the solutions found by them. There are two main explanations for the success of DEUM algorithms. Firstly, DEUM builds a model of fitness function to approximate the MRF. This contrasts with other EDAs, which build a model of selected solutions. This allows DEUM to use fitness in variation part of the evolution. Secondly, DEUM exploits the temperature coefficient in the Gibbs distribution to regulate the behaviour of the algorithm. In particular, with higher temperature, the distribution is closer to being uniform and with lower temperature it concentrates near some global optima. This gives DEUM an explicit control over the convergence of the algorithm, resulting in better optimisation.

Doctor of Philosophy / Department of Computing and Information Sciences / William Henry Hsu / Topic models are probabilistic models for discovering topical themes in collections of documents. In real world applications, these models provide us with the means of organizing what would otherwise be unstructured collections. They can help us cluster a huge collection into different topics or find a subset of the collection that resembles the topical theme found in an article at hand. The first wave of topic models developed were able to discover the prevailing topics in a big collection of documents spanning a period of time. It was later realized that these time-invariant models were not capable of modeling 1) the time varying number of topics they discover and 2) the time changing structure of these topics. Few models were developed to address this two deficiencies. The online-hierarchical Dirichlet process models the documents with a time varying number of topics. It varies the structure of the topics over time as well. However, it relies on document order, not timestamps to evolve the model over time. The continuous-time dynamic topic model evolves topic structure in continuous-time. However, it uses a fixed number of topics over time. In this dissertation, I present a model, the continuous-time infinite dynamic topic model, that combines the advantages of these two models 1) the online-hierarchical Dirichlet process, and 2) the continuous-time dynamic topic model. More specifically, the model I present is a probabilistic topic model that does the following: 1) it changes the number of topics over continuous time, and 2) it changes the topic structure over continuous-time. I compared the model I developed with the two other models with different setting values. The results obtained were favorable to my model and showed the need for having a model that has a continuous-time varying number of topics and topic structure.

Causal inference may help us to understand the underlying mechanisms and the risk factors of diseases. In Genetics, it is crucial to understand how the connectivity among variables is influenced by genetic and environmental factors. Family data have proven to be useful in elucidating genetic and environmental influences, however, few existing approaches are able of addressing structure learning of probabilistic graphical models (PGMs) and family data analysis jointly. We propose methodologies for learning, from observational Gaussian family data, the most likely PGM and its decomposition into genetic and environmental components. They were evaluated by a simulation study and applied to the Genetic Analysis Workshop 13 simulated data, which mimic the real Framingham Heart Study data, and to the metabolic syndrome phenotypes from the Baependi Heart Study. In neuroscience, one challenge consists in identifying interactions between functional brain networks (FBNs) - graphs. We propose a method to identify Granger causality among FBNs. We show the statistical power of the proposed method by simulations and its usefulness by two applications: the identification of Granger causality between the FBNs of two musicians playing a violin duo, and the identification of a differential connectivity from the right to the left brain hemispheres of autistic subjects. / Inferência causal pode nos ajudar a compreender melhor as relações de dependência direta entre variáveis e, assim, a identificar fatores de riscos de doenças. Em Genética, a análise de dados agrupados em famílias permite investigar influências genéticas e ambientais nas relações entre as variáveis. Neste trabalho, nós propomos métodos para aprender, a partir de dados Gaussianos agrupados em famílias, o mais provável modelo gráfico probabilístico (dirigido ou não dirigido) e também sua decomposição em dois componentes: genético e ambiental. Os métodos foram avaliados por simulações e aplicados tanto aos dados simulados do Genetic Analysis Workshop 13, que imitam características dos dados do Framingham Heart Study, como aos dados da síndrome metabólica do estudo Corações de Baependi. Em Neurociência, um desafio consiste em identificar interações entre redes funcionais cerebrais - grafos. Nós propomos um método que identifica causalidade de Granger entre grafos e, por meio de simulações, mostramos que o método tem alto poder estatístico. Além disso, mostramos sua utilidade por meio de duas aplicações: 1) identificação de causalidade de Granger entre as redes cerebrais de dois músicos enquanto tocam um dueto de violino e 2) identificação de conectividade diferencial do hemisfério cerebral direito para o esquerdo em indivíduos autistas.

After decades of research, a host of analytical difficulties is still hindering our understanding of the influences of the built form on travel. The main challenges are (a) assembling good quality data that reflects the majority of the known influences and that supports continuous monitoring, and (b) making sense methodologically of the many variables which strongly intercorrelate. This study uses the UK national travel survey (NTS) data that is among the most comprehensive of its form in the world. The fact that it has rarely been used so far for this purpose may be attributable to the methodological difficulties. This dissertation aims to develop a new analytical framework based on extended structural equation models (SEMs) in order to overcome some of the key methodological difficulties in quantifying the influences of the built form on travel, and in addition to provide a means to continuously monitor any changes in the effects over time. The analyses are focused on employed adults, because they are not only the biggest UK population segment with the highest per capita travel demand, but also the segment that are capable of adapting more rapidly to changing land use, built form and transport supply conditions. The research is pursued through three new models. Model 1 is a path diagram coupled with factor analyses, which estimates continuous, categorical and binary dependent variables. The model estimates the influences on travel distance, time and trip frequency by trip purpose while accounting for self-selection, spatial sorting, endogeneity of car ownership, and interactions among trip purposes. The results highlight stark differences among commuters, particularly the mobility disadvantages of women, part time and non-car owning workers even when they live in the most accessible urban areas. Model 2 incorporates latent categorisation analyses in order to identify a tangible typology of the built form and the associated variations in impacts on travel. Identifying NTS variables as descriptors for tangible built form categories provides an improved basis for investigating land use and transport planning interventions. The model reveals three distinct built form categories in the UK with striking variations in the patterns of influences. Model 3 further investigates the variations across the built form categories. The resulting random intercept SEM provides a more precise quantification of the influences of self-selection and spatial sorting across the built form categories for each socioeconomic group. Four research areas are highlighted for further studies: First, new preference, attitude and behavioural parameters may be introduced through incorporating non-NTS behavioural surveys; Second, the new SEMs provide a basis for incorporating choice modelling where the utility function is defined with direct, indirect and latent variables; Third, conceptual and methodological developments – such as non-parametric latent class analysis, allow expanding the current model to monitor changes in travel behaviour as and when new NTS or non NTS data become available. Fourth, the robustness of the inferences regarding causal or directional influences may require further quantification through designing new panel data sets, building on the findings above.

La prédiction et l'inférence sur des données temporelles sont très souvent effectuées en utilisant uniquement les séries temporelles. Nous sommes convaincus que ces tâches pourraient tirer parti de l'utilisation des métadonnées contextuelles associées aux séries temporelles, telles que l'emplacement, le type, etc. Réciproquement, les tâches de prédiction et d'inférence sur les métadonnées pourraient bénéficier des informations contenues dans les séries temporelles. Cependant, il n'existe pas de méthode standard pour modéliser conjointement les données de séries temporelles et les métadonnées descriptives. De plus, les métadonnées contiennent fréquemment des informations hautement corrélées ou redondantes et peuvent contenir des erreurs et des valeurs manquantes.Nous examinons d’abord le problème de l’apprentissage de la structure graphique probabiliste inhérente aux métadonnées en tant que réseau Bayésien. Ceci présente deux avantages principaux: (i) une fois structurées en tant que modèle graphique, les métadonnées sont plus faciles à utiliser pour améliorer les tâches sur les données temporelles et (ii) le modèle appris permet des tâches d'inférence sur les métadonnées uniquement, telles que l'imputation de données manquantes. Cependant, l'apprentissage de la structure de réseau Bayésien est un défi mathématique conséquent, impliquant un problème d'optimisation NP-difficile. Pour faire face à ce problème, nous présentons un algorithme d'apprentissage de structure sur mesure, inspiré de nouveaux résultats théoriques, qui exploite les dépendances (quasi)-déterministes généralement présentes dans les métadonnées descriptives. Cet algorithme est testé sur de nombreux jeux de données de référence et sur certains jeux de métadonnées industriels contenant des relations déterministes. Dans les deux cas, il s'est avéré nettement plus rapide que l'état de la l'art, et a même trouvé des structures plus performantes sur des données industrielles. De plus, les réseaux Bayésiens appris sont toujours plus parcimonieux et donc plus lisibles.Nous nous intéressons ensuite à la conception d'un modèle qui inclut à la fois des (méta)données statiques et des données temporelles. En nous inspirant des modèles graphiques probabilistes pour les données temporelles (réseaux Bayésiens dynamiques) et de notre approche pour la modélisation des métadonnées, nous présentons une méthodologie générale pour modéliser conjointement les métadonnées et les données temporelles sous forme de réseaux Bayésiens hybrides statiques-dynamiques. Nous proposons deux algorithmes principaux associés à cette représentation: (i) un algorithme d'apprentissage qui, bien qu'optimisé pour les données industrielles, reste généralisable à toute tâche de fusion de données statiques et dynamiques, et (ii) un algorithme d'inférence permettant les d'effectuer à la fois des requêtes sur des données temporelles ou statiques uniquement, et des requêtes utilisant ces deux types de données.%Nous fournissons ensuite des résultats sur diverses applications inter-domaines telles que les prévisions, le réapprovisionnement en métadonnées à partir de séries chronologiques et l’analyse de dépendance d’alarmes en utilisant les données de certains cas d’utilisation difficiles de Schneider Electric.Enfin, nous approfondissons certaines des notions introduites au cours de la thèse, et notamment la façon de mesurer la performance en généralisation d’un réseau Bayésien par un score inspiré de la procédure de validation croisée provenant de l’apprentissage automatique supervisé. Nous proposons également des extensions diverses aux algorithmes et aux résultats théoriques présentés dans les chapitres précédents, et formulons quelques perspectives de recherche. / Prediction and inference on temporal data is very frequently performed using timeseries data alone. We believe that these tasks could benefit from leveraging the contextual metadata associated to timeseries - such as location, type, etc. Conversely, tasks involving prediction and inference on metadata could benefit from information held within timeseries. However, there exists no standard way of jointly modeling both timeseries data and descriptive metadata. Moreover, metadata frequently contains highly correlated or redundant information, and may contain errors and missing values.We first consider the problem of learning the inherent probabilistic graphical structure of metadata as a Bayesian Network. This has two main benefits: (i) once structured as a graphical model, metadata is easier to use in order to improve tasks on temporal data and (ii) the learned model enables inference tasks on metadata alone, such as missing data imputation. However, Bayesian network structure learning is a tremendous mathematical challenge, that involves a NP-Hard optimization problem. We present a tailor-made structure learning algorithm, inspired from novel theoretical results, that exploits (quasi)-determinist dependencies that are typically present in descriptive metadata. This algorithm is tested on numerous benchmark datasets and some industrial metadatasets containing deterministic relationships. In both cases it proved to be significantly faster than state of the art, and even found more performant structures on industrial data. Moreover, learned Bayesian networks are consistently sparser and therefore more readable.We then focus on designing a model that includes both static (meta)data and dynamic data. Taking inspiration from state of the art probabilistic graphical models for temporal data (Dynamic Bayesian Networks) and from our previously described approach for metadata modeling, we present a general methodology to jointly model metadata and temporal data as a hybrid static-dynamic Bayesian network. We propose two main algorithms associated to this representation: (i) a learning algorithm, which while being optimized for industrial data, is still generalizable to any task of static and dynamic data fusion, and (ii) an inference algorithm, enabling both usual tasks on temporal or static data alone, and tasks using the two types of data.%We then provide results on diverse cross-field applications such as forecasting, metadata replenishment from timeseries and alarms dependency analysis using data from some of Schneider Electric’s challenging use-cases.Finally, we discuss some of the notions introduced during the thesis, including ways to measure the generalization performance of a Bayesian network by a score inspired from the cross-validation procedure from supervised machine learning. We also propose various extensions to the algorithms and theoretical results presented in the previous chapters, and formulate some research perspectives.

[pt] O crescimento da população do planeta tem aumentado continuamente a demanda por produtos agrícolas. Assim, a informação quanto a áreas cultivadas e estimativas de produção se tornam cada vez mais importantes. Técnicas baseadas em imagens satelitais constituem uma das opções mais atrativas para o monitoramento agrícola sobre grandes áreas. A maior parte dos trabalhos científicos voltados a esta aplicação foram desenvolvidos para regiões temperadas do planeta, que apresentam um dinâmica muito mais simples da que se tem em regiões tropicais. Neste contexto, a presente tese propõe um novo método automático baseado em Campos Aleatórios Condicionais (CRF) para o reconhecimento de culturas agrícolas em regiões tropicais a partir de sequências de imagens multi-temporais e multiresolução produzidas por diferentes sensores orbitais. Experimentos foram realizados para validar diversas variantes do método proposto. Utilizaramse bases de dados públicas de duas regiões do Brasil que compreendem sequências de imagens óticas e de radar com diferentes resoluções espaciais. Os experimentos realizados demonstraram que o método proposto atingiu acurácias maiores do que métodos baseados em uma única imagem ou sensor. Particularmente, notou-se a redução do efeito sal-e-pimenta nos mapas gerados devido, mormente, à capacidade do método de capturar informação contextual. / [en] The earth population growth has continuously increased the demand for agricultural production. Thus, acreage and crop yield information become increasingly important. Techniques based on satellite images are one of the most attractive options for agricultural monitoring over large areas. Most of the scientific works on this application were developed for temperate regions of the planet, which present a much simpler dynamics than those in tropical regions. In this context, the present thesis proposes a new automatic method based on Conditional Random Fields (CRF) for the crop recognition in tropical regions from multi-temporal and multi-resolution image sequences from orbital multi-sensors. Experiments were performed to validate several variants of the proposed method. We used public databases from two regions of Brazil that comprise sequences of optical and radar images with different spatial resolutions. The experiments demonstrated that the proposed method achieved a higher accuracy than methods based on a single image or sensor. Particularly, the reduction of the salt-and-pepper effect in the generated maps was noticed due, mainly, to the capacity of the method to capture contextual information.