Hypothesis-free learning is increasingly popular given the large amounts of data becoming available. Structure learning, a hypothesis-free approach, of graphical models is a field of growing interest due to the power of such models and lack of domain knowledge when applied on complex real-world data. State-of-the-art techniques improve on scalability of structure learning, which is often characterized by a large problem space. Nonetheless, these techniques still suffer computational bottlenecks that are yet to be approached. In this work, we focus on two popular models: dynamical linear systems and Markov random fields. For each case, we investigate major computational bottlenecks of baseline learning techniques. Next, we propose two frameworks that provide higher scalability using appropriate problem reformulation and efficient structure based heuristics. We perform experiments on synthetic and real data to validate our theoretical analysis. Current results show that we obtain a quality similar to expensive baseline techniques but with higher scalability. / Master of Science / Structure learning of graphical models is the process of understanding the interactions and influence between the variables of a given system. A few examples of such systems are road traffic systems, stock markets, and social networks. Learning the structure uncovers the invisible inter-variables relationships that govern their evolution. This process is key to qualitative analysis and forecasting. A classic approach to obtain the structure is through domain experts. For example, a financial expert could draw a graphical structure that encodes the relationships between different software companies based on his knowledge in the field. However, the absence of domain experts in the case of complex and heterogeneous systems has been a great motivation for the field of data driven, hypothesis free structure learning. Current techniques produce good results but unfortunately require a high computational cost and are often slow to execute. In this work, we focus on two popular graphical models that require computationally expensive structure learning methods. We first propose theoretical analysis of the high computational cost of current techniques. Next, we propose a novel approach for each model. Our proposed methods perform structure learning faster than baseline methods and provide a higher scalability to systems of large number of variables and large datasets as shown in our theoretical analysis and experimental results.

Uma das etapas mais importantes da análise de imagens e, que conta com uma enorme quantidade de aplicações, é a segmentação. No entanto, uma boa parte das técnicas tradicionais apresenta alto custo computacional, dificultando sua aplicação em imagens de alta resolução como, por exemplo, as imagens de ninhais de aves do Pantanal que também serão analisadas neste trabalho. Diante disso, é proposta uma nova abordagem de segmentação que combina algoritmos de detecção de comunidades, pertencentes à teoria das redes complexas, com técnicas de extração de superpixels. Tal abordagem é capaz de segmentar imagens de alta resolução mantendo o compromisso entre acurácia e tempo de processamento. Além disso, como as imagens de ninhais analisadas apresentam características peculiares que podem ser mais bem tratadas por técnicas de segmentação por textura, a técnica baseada em Markov Random Fields (MRF) é proposta, como um complemento à abordagem de segmentação inicial, para realizar a identificação final das aves. Por fim, devido à importância de avaliar quantitativamente a qualidade das segmentações obtidas, um nova métrica de avaliação baseada em ground-truth foi desenvolvida, sendo de grande importância para a área. Este trabalho contribuiu para o avanço do estado da arte das técnicas de segmentação de imagens de alta resolução, aprimorando e desenvolvendo métodos baseados na combinação de redes complexas com superpixels, os quais alcançaram resultados satisfatórios com baixo tempo de processamento. Além disso, uma importante contribuição referente ao censo demográfico de aves por meio da análise de imagens aéreas de ninhais foi viabilizada por meio da aplicação da técnica de segmentação MRF. / Segmentation is one of the most important steps in image analysis with a large range of applications. However, some traditional techniques exhibit high computational costs, hindering their application in high resolution images such as the images of birds nests from Pantanal, one of Brazilian most important wetlands. Therefore, we propose a new segmentation approach that combines community detection algorithms, originated from the theory of the complex networks, with superpixels extraction techniques. This approach is capable of segmenting high resolution images while maintaining the trade-off between accuracy and processing time. Moreover, as the nest images exhibit peculiar characteristics that can be better dealt with texture segmentation techniques, the Markov Random Fields (MRF) technique is proposed, as a complement to the initial approach, to perform the final identification of the birds. Finally, due to the importance of the quantitatively evaluation of the segmentation quality, a new evaluation metric based on ground-truth was developed, being of great importance to the segmentation field. This work contributed to the state of art of high resolution images segmentation techniques, improving and developing methods based on combination of complex networks and superpixels, which generated satisfactory results within low processing time. Moreover, an important contribution for the birds census by the analysis of aerial images of birds nests was made possible by application of the MRF technique.

En kunskapsgraf lagrar information från webben i form av relationer mellan olika entiteter. En kunskapsgrafs kvalité bestäms av hur komplett den är och dess noggrannhet. Dessvärre har många nuvarande kunskapsgrafer brister i form av saknad fakta och inkorrekt information. Nuvarande lösningar av länkförutsägelser mellan entiteter har problem med skalbarhet och hög arbetskostnad. Denna uppsats föreslår ett deklarativt regelbaserat probabilistiskt ramverk för att utföra länkförutsägelse. Systemet involverar en regelutvinnande modell till ett “hinge-loss Markov random fields” för att föreslå länkar. Vidare utvecklades tre strategier för regeloptimering för att förbättra reglernas kvalité. Jämfört med tidigare lösningar så bidrar detta arbete till att drastiskt reducera arbetskostnader och en mer spårbar modell. Varje metod har utvärderas med precision och F-värde på NELL och Freebase15k. Det visar sig att strategin för regeloptimering presterade bäst. MAP-uppskattningen för den bästa modellen på NELL är 0.754, vilket är bättre än en nuvarande spjutspetsteknologi graphical model(0.306). F-värdet för den bästa modellen på Freebase15k är 0.709. / The knowledge graph stores factual information from the web in form of relationships between entities. The quality of a knowledge graph is determined by its completeness and accuracy. However, most current knowledge graphs often miss facts or have incorrect information. Current link prediction solutions have problems of scalability and high labor costs. This thesis proposed a declarative rule-based probabilistic framework to perform link prediction. The system incorporates a rule-mining model into a hingeloss Markov random fields to infer links. Moreover, three rule optimization strategies were developed to improve the quality of rules. Compared with previous solutions, this work dramatically reduces manual costs and provides a more tractable model. Each proposed method has been evaluated with Average Precision or F-score on NELL and Freebase15k. It turns out that the rule optimization strategy performs the best. The MAP of the best model on NELL is 0.754, better than a state-of-the-art graphical model (0.306). The F-score of the best model on Freebase15k is 0.709.

Uma das etapas mais importantes da análise de imagens e, que conta com uma enorme quantidade de aplicações, é a segmentação. No entanto, uma boa parte das técnicas tradicionais apresenta alto custo computacional, dificultando sua aplicação em imagens de alta resolução como, por exemplo, as imagens de ninhais de aves do Pantanal que também serão analisadas neste trabalho. Diante disso, é proposta uma nova abordagem de segmentação que combina algoritmos de detecção de comunidades, pertencentes à teoria das redes complexas, com técnicas de extração de superpixels. Tal abordagem é capaz de segmentar imagens de alta resolução mantendo o compromisso entre acurácia e tempo de processamento. Além disso, como as imagens de ninhais analisadas apresentam características peculiares que podem ser mais bem tratadas por técnicas de segmentação por textura, a técnica baseada em Markov Random Fields (MRF) é proposta, como um complemento à abordagem de segmentação inicial, para realizar a identificação final das aves. Por fim, devido à importância de avaliar quantitativamente a qualidade das segmentações obtidas, um nova métrica de avaliação baseada em ground-truth foi desenvolvida, sendo de grande importância para a área. Este trabalho contribuiu para o avanço do estado da arte das técnicas de segmentação de imagens de alta resolução, aprimorando e desenvolvendo métodos baseados na combinação de redes complexas com superpixels, os quais alcançaram resultados satisfatórios com baixo tempo de processamento. Além disso, uma importante contribuição referente ao censo demográfico de aves por meio da análise de imagens aéreas de ninhais foi viabilizada por meio da aplicação da técnica de segmentação MRF. / Segmentation is one of the most important steps in image analysis with a large range of applications. However, some traditional techniques exhibit high computational costs, hindering their application in high resolution images such as the images of birds nests from Pantanal, one of Brazilian most important wetlands. Therefore, we propose a new segmentation approach that combines community detection algorithms, originated from the theory of the complex networks, with superpixels extraction techniques. This approach is capable of segmenting high resolution images while maintaining the trade-off between accuracy and processing time. Moreover, as the nest images exhibit peculiar characteristics that can be better dealt with texture segmentation techniques, the Markov Random Fields (MRF) technique is proposed, as a complement to the initial approach, to perform the final identification of the birds. Finally, due to the importance of the quantitatively evaluation of the segmentation quality, a new evaluation metric based on ground-truth was developed, being of great importance to the segmentation field. This work contributed to the state of art of high resolution images segmentation techniques, improving and developing methods based on combination of complex networks and superpixels, which generated satisfactory results within low processing time. Moreover, an important contribution for the birds census by the analysis of aerial images of birds nests was made possible by application of the MRF technique.

Thesis (MScEng (Electrical and Electronic Engineering))--University of Stellenbosch, 2010. / ENGLISH ABSTRACT: In this thesis we present GrMPy, a library of classes and functions implemented in Python, designed for implementing graphical models. GrMPy supports both undirected and directed models, exact and approximate probabilistic inference, and parameter estimation from complete and incomplete data. In this thesis we outline the necessary theory required to understand the tools implemented within GrMPy as well as provide pseudo-code algorithms that illustrate how GrMPy is implemented. / AFRIKAANSE OPSOMMING: In hierdie verhandeling bied ons GrMPy aan,'n biblioteek van klasse en funksies wat Python geim- plimenteer word en ontwerp is vir die implimentering van grafiese modelle. GrMPy ondersteun beide gerigte en ongerigte modelle, presies eenbenaderde moontlike gevolgtrekkings en parameterskat- tings van volledige en onvolledige inligting. In hierdie verhandeling beskryf ons die nodige teorie wat benodig word om die hulpmiddels wat binne GrMPy geimplimenteer word te verstaan sowel as die pseudo-kodealgoritmes wat illustreer hoe GrMPy geimplimenteer is.

Nous présentons une approche de reconnaissance d'écriture manuscrite à partir de champs de Markov cachés et fondée sur une analyse entièrement bidimensionnelle de l'écriture. Son originalité réside dans la combinaison d'une analyse fenêtrée de l'image, d'une modélisation markovienne et dans la mise en oeuvre de la programmation dynamique 2D qui permet un décodage rapide et optimal des champs de Markov. Un aspect important de ces travaux est la méthodologie de développement employée qui est centrée sur l'évaluation systématique des apports algorithmiques et des paramètres utilisés. Ces algorithmes sont en partie empruntés aux techniques utilisées dans le domaine de la reconnaissance de la parole et sont très génériques.<br /><br />L'approche proposée est validée sur deux applications correspondant à des bases de données standard et librement disponibles. L'application de cette méthode extrêmement générique à une tâche de reconnaissance de chiffres manuscrits a permis d'obtenir des résultats comparables à ceux de l'état de l'art. L'application à une tâche de reconnaissance de mots manuscrits a permis de confirmer que l'extension de cette approche à des tâches plus complexes était naturelle.<br /><br />L'ensemble de cette recherche a démontré la validité de l'approche développée qui apparaît comme candidate au statut d'approche standard pour plusieurs problèmes de vision. En outre, elle ouvre la voie à de très nombreux développements concernant la tâche de traitement de l'écriture manuscrite et des améliorations significatives pourraient encore être apportées en recourant à d'autres principes issus du traitement de la parole et du langage. D'autres tâches comme la segmentation d'image devraient tirer avantage de la robustesse et de la faculté d'apprentissage de la modélisation que nous proposons.

The topic of Markov Random Fields (MRFs) has been well studied in the past, and has found practical use in various image processing, and machine learning applications. Where coding is concerned, MRF specific schemes have been largely unexplored. In this thesis, an overview is given of recent developments and challenges in the lossless coding of MRFs. Specifically, we concentrate on difficulties caused by computational intractability due to the partition function of the MRF. One proposed solution to this problem is to segment the MRF with a cutset, and encode the components separately. Using this method, arithmetic coding is possible via the Belief Propagation (BP) algorithm. We consider two cases of the BP algorithm: MRFs with only simple cliques, and MRFs with complex cliques. In the latter case, we study a minimum radius condition requirement for ensuring that all cliques are accounted for during coding. This condition also simplifies the process of conditioning on observed sites. Finally, using these results, we develop a systematic procedure of clustering and choosing cutsets. / Thesis (Master, Mathematics & Statistics) -- Queen's University, 2013-08-12 14:50:00.596

This thesis is devoted to the development of graph-based methods that address several of the most fundamental computer vision problems, such as segmentation, tracking, shape matching and 3D model inference. The first contribution of this thesis is a unified, single-shot optimization framework for simultaneous segmentation, depth ordering and multi-object tracking from monocular video sequences using a pairwise Markov Random Field (MRF). This is achieved through a novel 2.5D layered model where object-level and pixel-level representations are seamlessly combined through local constraints. Towards introducing high-level knowledge, such as shape priors, we then studied the problem of non-rigid 3D surface matching. The second contribution of this thesis consists of a higher-order graph matching formulation that encodes various measurements of geometric/appearance similarities and intrinsic deformation errors. As the third contribution of this thesis, higher-order interactions were further considered to build pose-invariant statistical shape priors and were exploited for the development of a novel approach for knowledge-based 3D segmentation in medical imaging which is invariant to the global pose and the initialization of the shape model. The last contribution of this thesis aimed to partially address the influence of camera pose in visual perception. To this end, we introduced a unified paradigm for 3D landmark model inference from monocular 2D images to simultaneously determine both the optimal 3D model and the corresponding 2D projections without explicit estimation of the camera viewpoint, which is also able to deal with misdetections/occlusions

In this dissertation, we propose three methods to solve the problems of interactive image segmentation, video completion, and image denoising, which are all formulated as MRF-based energy minimization problems. In our algorithms, different MRF-based energy functions with particular techniques according to the characteristics of different tasks are designed to well fit the problems. With the energy functions, different optimization schemes are proposed to find the optimal results in these applications. In interactive image segmentation, an iterative optimization based framework is proposed, where in each iteration an MRF-based energy function incorporating an estimated initial probabilistic map of the image is optimized with a relaxed global optimal solution. In video completion, a well-defined MRF energy function involving both spatial and temporal coherence relationship is constructed based on the local motions calculated in the first step of the algorithm. A hierarchical belief propagation optimization scheme is proposed to efficiently solve the problem. In image denoising, label relaxation based optimization on a Gaussian MRF energy is used to achieve the global optimal closed form solution. / Many problems in computer vision involve assigning each pixel a label, which represents some spatially varying quantity such as image intensity in image denoising or object index label in image segmentation. In general, such quantities in image processing tend to be spatially piecewise smooth, since they vary smoothly in the object surface and change dramatically at object boundaries, while in video processing, additional temporal smoothness is satisfied as the corresponding pixels in different frames should have similar labels. Markov random field (MRF) models provide a robust and unified framework for many image and video applications. The framework can be elegantly expressed as an MRF-based energy minimization problem, where two penalty terms are defined with different forms. Many approaches have been proposed to solve the MRF-based energy optimization problem, such as simulated annealing, iterated conditional modes, graph cuts, and belief propagation. / Promising results obtained by the proposed algorithms, with both quantitative and qualitative comparisons to the state-of-the-art methods, demonstrate the effectiveness of our algorithms in these image and video processing applications. / Liu, Ming. / Adviser: Xiaoou Tang. / Source: Dissertation Abstracts International, Volume: 72-04, Section: B, page: . / Thesis (Ph.D.)--Chinese University of Hong Kong, 2010. / Includes bibliographical references (leaves 79-89). / Electronic reproduction. Hong Kong : Chinese University of Hong Kong, [2012] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Electronic reproduction. Ann Arbor, MI : ProQuest Information and Learning Company, [200-] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Electronic reproduction. Ann Arbor, MI : ProQuest Information and Learning Company, [200-] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Abstract also in Chinese.

A long term record of the earth's vegetation is important in studies of global climate change. Over the last three decades, multiple data sets on vegetation have been collected using different satellite-based sensors. There is a need for methods that combine these data into a long term earth system data record. The Advanced Very High Resolution Radiometer (AVHRR) has provided reflectance measures of the entire earth since 1978. Physical and statistical models have been used to improve the consistency and reliability of this record. The Moderated Resolution Imaging Spectroradiometer (MODIS) has provided measurements with superior radiometric properties and geolocation accuracy. However, this record is available only since 2000. In this thesis, we perform statistical calibration of AVHRR to MODIS. We aim to: (1) fill in gaps in the ongoing MODIS record; (2) extend MODIS values back to 1982. We propose Bayesian mixed models to predict MODIS values using snow cover and AVHRR values as covariates. Random effects are used to account for spatiotemporal correlation in the data. We estimate the parameters based on the data after 2000, using Markov chain Monte Carlo methods. We then back-predict MODIS data between 1978 and 1999, using the posterior samples of the parameter estimates. We develop new Conditional Autoregressive (CAR) models for seasonal data. We also develop new sampling methods for CAR models. Our approach enables filling in gaps in the MODIS record and back-predicting these values to construct a consistent historical record. The Bayesian framework incorporates multiple sources of variation in estimating the accuracy of the obtained data. The approach is illustrated using vegetation data over a region in Minnesota.