In the past years, several network reconstruction methods modeled as Gaussian Graphical Model in high dimensional settings where proposed. In this work we will analyze three different methods, the Graphical Lasso (GLasso), Graphical Ridge (GGMridge) and a novel method called LPC, or Local Partial Correlation. The evaluation will be performed in high dimensional data generated from different simulated random graph structures (Erdos-Renyi, Barabasi-Albert, Watts-Strogatz ), using Receiver Operating Characteristic or ROC curve. We will also apply the methods in the reconstruction of genetic co-expression network for the differentially expressed genes in cervical cancer tumors. / Vários métodos tem sido propostos para a reconstrução de redes em alta dimensão, que e tratada como um Modelo Gráfico Gaussiano. Neste trabalho vamos analisar três métodos diferentes, o método Graphical Lasso (GLasso), Graphical Ridge (GGMridge) e um novo método chamado LPC, ou Correlação Parcial Local. A avaliação será realizada em dados de alta dimensão, gerados a partir de grafos aleatórios (Erdos-Renyi, Barabasi-Albert, Watts-Strogatz ), usando Receptor de Operação Característica, ou curva ROC. Aplicaremos também os metidos apresentados, na reconstrução da rede de co-expressão gênica para tumores de câncer cervical.

In practice, we often meet data in the form of observations of several variables at various points in time. These data are called time series. We present various approaches in time series analysis; graphical models, vector autoregres- sive models and vector moving-average models. We try to get information about mutual relationship of the variables and then to model their behavior. The used techniques are illustrated on log returns of monthly average exchange rates. The programs are processed in the software Mathematica 7 and can be found on the CD. 1

Convex optimization has developed a wide variety of useful tools critical to many applications in machine learning. However, unlike linear and quadratic programming, general convex solvers have not yet reached sufficient maturity to fully decouple the convex programming model from the numerical algorithms required for implementation. Especially as datasets grow in size, there is a significant gap in speed and scalability between general solvers and specialized algorithms. This thesis addresses this gap with a new model for convex programming based on an intermediate representation of convex problems as a sum of functions with efficient proximal operators. This representation serves two purposes: 1) many problems can be expressed in terms of functions with simple proximal operators, and 2) the proximal operator form serves as a general interface to any specialized algorithm that can incorporate additional `2-regularization. On a single CPU core, numerical results demonstrate that the prox-affine form results in significantly faster algorithms than existing general solvers based on conic forms. In addition, splitting problems into separable sums is attractive from the perspective of distributing solver work amongst multiple cores and machines. We apply large-scale convex programming to several problems arising from building the next-generation, information-enabled electrical grid. In these problems (as is common in many domains) large, high-dimensional datasets present opportunities for novel data-driven solutions. We present approaches based on convex models for several problems: probabilistic forecasting of electricity generation and demand, preventing failures in microgrids and source separation for whole-home energy disaggregation.

No description available.

One of the most important and challenging ``knowledge extraction' tasks in bioinformatics is the reverse engineering of gene regulatory networks (GRNs) from DNA microarray gene expression data. Indeed, as a result of the development of high-throughput data-collection techniques, biology is experiencing a data flood phenomenon that pushes biologists toward a new view of biology--systems biology--that aims at system-level understanding of biological systems. Unfortunately, even for small model organisms such as the yeast Saccharomyces cerevisiae, the number p of genes is much larger than the number n of expression data samples. The dimensionality issue induced by this ``small n, large p' data setting renders standard statistical learning methods inadequate. Restricting the complexity of the models enables to deal with this serious impediment. Indeed, by introducing (a priori undesirable) bias in the model selection procedure, one reduces the variance of the selected model thereby increasing its accuracy. Gaussian graphical models (GGMs) have proven to be a very powerful formalism to infer GRNs from expression data. Standard GGM selection techniques can unfortunately not be used in the ``small n, large p' data setting. One way to overcome this issue is to resort to regularization. In particular, shrinkage estimators of the covariance matrix--required to infer GGMs--have proven to be very effective. Our first contribution consists in a new shrinkage estimator that improves upon existing ones through the use of a Monte Carlo (parametric bootstrap) procedure. Another approach to GGM selection in the ``small n, large p' data setting consists in reverse engineering limited-order partial correlation graphs (q-partial correlation graphs) to approximate GGMs. Our second contribution consists in an inference algorithm, the q-nested procedure, that builds a sequence of nested q-partial correlation graphs to take advantage of the smaller order graphs' topology to infer higher order graphs. This allows us to significantly speed up the inference of such graphs and to avoid problems related to multiple testing. Consequently, we are able to consider higher order graphs, thereby increasing the accuracy of the inferred graphs. Another important challenge in bioinformatics is the prediction of gene function. An example of such a prediction task is the identification of genes that are targets of the nitrogen catabolite repression (NCR) selection mechanism in the yeast Saccharomyces cerevisiae. The study of model organisms such as Saccharomyces cerevisiae is indispensable for the understanding of more complex organisms. Our third contribution consists in extending the standard two-class classification approach by enriching the set of variables and comparing several feature selection techniques and classification algorithms. Finally, our fourth contribution formulates the prediction of NCR target genes as a network inference task. We use GGM selection to infer multivariate dependencies between genes, and, starting from a set of genes known to be sensitive to NCR, we classify the remaining genes. We hence avoid problems related to the choice of a negative training set and take advantage of the robustness of GGM selection techniques in the ``small n, large p' data setting.

Grafinių modelių projektavimas yra plačiai naudojamas tiek mokslo, tiek verslo srytyse. Pasaulyje naudojama įvairių kalbų, skirtų tiek sistemų architektūrų, tiek verslo procesų projektavimui. Daugumai kalbų yra sukurta įvairių įrankių, leidžiančių jų naudotojams projektuoti įvairius procesus ar statines sistemas. Vienai labiausiai paplitusių kalbų (UML) trūksta metodikos ir įrankių, gebančių korektiškai perteikti natūralia kalba sistemų architektų aprašytus grafinius modelius asmenims, mažai kvalifikuotiems grafinių modelių sudaryme, skaityme. Perteikimas tuo natūralesnis ir labiau suprantamesnis, kuo jis artimesnis natūraliai kalbai. Yra metodikų ir įrankių atliekančių grafinio modelio verbalizavimą, tačiau nėra koncentruotų ties diagramomis UML kalba, kurios geba formuoti ne tik statiką, bet ir dinamiką. Pagrindinis darbo tikslas yra sukurti metodiką ir realizuoti įrankį, kuris gebėtų grafinį modelį išreikštą UML kalba performuluoti natūralia kalba. / The graphical model architecture design is widely used for scientific and enterprise purposes. There are many languages concentrated on enterprise processes and static systems designing. One of the most popular modeling language (UML) is missing methodology and tools suitable for correct reformulation of graphical models (formulated by the UML) in natural language. The main purpose of the graphical model reformulation in natural language is to make models easier to understand for people whose are not specialized in UML. Methodology and tool which is capable of reformulating graphical models in natural language already exists, but it isn’t concentrated on UML or capable of reformulating static and dynamic processes. The main goal of this work is to define a methodology and implement a tool, which would be capable of translating the graphical UML model to a natural language text.

In today’s machine learning research, probabilistic graphical models are used extensively to model complicated systems with uncertainty, to help understanding of the problems, and to help inference and predict unknown events. For inference tasks, exact inference methods such as junction tree algorithms exist, but they suffer from exponential growth of cluster size and thus is not able to handle large and highly connected graphs. Approximate inference methods do not try to find exact probabilities, but rather give results that improve as algorithm runs. Gibbs sampling, as one of the approximate inference methods, has gained lots of traction and is used extensively in inference tasks, due to its ease of understanding and implementation. However, as problem size grows, even the faster algorithm needs a speed boost to meet application requirement. The number of variables in an application graphical model can range from tens of thousands to billions, depending on problem domain. The original sequential Gibbs sampling may not return satisfactory result in limited time. Thus, in this thesis, we investigate in ways to speed up Gibbs sampling. We will study ways to do better initialization, blocking variables to be sampled together, as well as using simulated annealing. These are the methods that modifies the algorithm itself. We will also investigate in ways to parallelize the algorithm. An algorithm is parallelizable if some steps do not depend on other steps, and we will find out such dependency in Gibbs sampling. We will discuss how the choice of different hardware and software architecture will affect the parallelization result. We will use optical flow problem as an example to demonstrate the various speed up methods we investigated. An optical flow method tries to find out the movements of small image patches between two images in a temporal sequence. We demonstrate how we can model it using probabilistic graphical model, and solve it using Gibbs sampling. The result of using sequential Gibbs sampling is demonstrated, with comparisons from using various speed up methods and other optical flow methods.

Objective: This thesis focuses on design and evaluation of a disease prediction system that be able to detect hidden and upcoming diseases of an individual. Unlike previous works that has typically relied on precise medical examinations to extract symptoms and risk factors for computing probability of occurrence of a disease, the proposed disease prediction system is based on similar patterns of disease comorbidity in population and the individual to evaluate the risk of a disease. Methods: We combine three machine learning algorithms to construct the prediction system: an item based recommendation system, a Bayesian graphical model and a rule based recommender. We also propose multiple similarity measures for the recommendation system, each useful in a particular condition. We finally show how best values of parameters of the system can be derived from optimization of cost function and ROC curve. Results: A permutation test is designed to evaluate accuracy of the prediction system accurately. Results showed considerable advantage of the proposed system in compare to an item based recommendation system and improvements of prediction if system is trained for each specific gender and race. Conclusion: The proposed system has been shown to be a competent method in accurately identifying potential diseases in patients with multiple diseases, just based on their disease records. The procedure also contains novel soft computing and machine learning ideas that can be used in prediction problems. The proposed system has the possibility of using more complex datasets that include timeline of diseases, disease networks and social network. This makes it an even more capable platform for disease prediction. Hence, this thesis contributes to improvement of the disease prediction field. / Graduate / 0800 / 0766 / 0984 / mehrdadmansouri@yahoo.com

Graphical models in statistics and econometrics provide capability to describe causal relations using causal graph in classical regression analysis and others econometric tools. Goal of this thesis is description of causal modelling of time series with help of structural models of vector autoregression. There is description of procedure of building structural VAR model, principle of graphical models and building model for causal dependence analysis. For purpose of comparison there are used data from both USA and Czech Republic and comparison of similar models for both countries is presented. Best models are then selected, to show causal relations between macroeconomic variables. For purpose of analysis, impulse-response functions are used to show impact of demand shock on GDP and other macro indicators.

Existing approaches to continuous optimization are essentially mechanisms for deciding which locations should be sampled in order to obtain information about a target function's global optimum. These methods, while often effective in particular domains, generally base their decisions on heuristics developed in consideration of ill-defined desiderata rather than on explicitly defined goals or models of the available information that may be used to achieve them. The problem of numerical optimization is essentially one of deciding what information to gather, then using that information to infer the location of the global optimum. That being the case, it makes sense to model the problem using the language of decision theory and Bayesian inference. The contribution of this work is precisely such a model of the optimization problem, a model that explicitly describes information relationships, admits clear expression of the target function class as dictated by No Free Lunch, and makes rational and mathematically principled use of utility and cost. The result is an algorithm that displays surprisingly sophisticated behavior when supplied with simple and straightforward declarations of the function class and the utilities and costs of sampling. In short, this work intimates that continuous optimization is equivalent to statistical inference and decision theory, and the result of viewing the problem in this way has concrete theoretical and practical benefits.