As scientific simulations get increasingly complex, so do the requirements of how to deal with the data that is produced. Few scientists and engineers today are satisfied with just looking at streams of numbers; we require graphical visualizations to better understand their meaning. The traditional method of visualization has been to save the simulation's results to a file, then load that file up in another program (eg. Microsoft Excel) for post-processing. Although post-processing data to produce visualizations may be sufficient for some simple simulations, a modern simulation designer usually wants more out of their visualization. Perhaps they want the visualization to be a 3D plot of an interplanetary trajectory, with the ability to zoom, pan, and rotate the scene interactively. Until now, doing so has required the designer to become adept at computer graphics, which is a feat that almost no scientist or engineer has the time to attempt. The research undertaken here introduces an architecture by which a simulation programmer can easily add interactive 3D visualizations to their simulations. This architecture has several benefits over existing visualization packages, the biggest one being that no knowledge of computer graphics is required to use the it in one's own simulations. Another benefit is that the resulting visualization is interactive by default, without any extra programming required on the part of the simulation designer. This thesis begins by introducing the theory behind how scientific simulations want to visualize data. Common aspects of all simulations are identified, and are used to develop a common "visualization language" that can be used by any simulation designer to specify what they want to visualize. The second part of the thesis specifies a particular implementation of this visualization language, called OpenFrames. Open- Frames is a library of functions that can be called from C, C++, or FORTRAN, and automatically implements the visualization specified by the designer.

Στόχος της παρούσας διπλωματικής εργασίας ήταν η μελέτη, ο σχεδιασμός και η υλοποίηση μίας Γραφικής Διεπαφής μέσω της οποίας μπορεί ο χρήστης αυτής να επεξεργαστεί μία ψηφιακή εικόνα τομογραφίας ανθρώπινου δακτύλου. Στο πρώτο κεφάλαιο γίνεται μία εισαγωγή στην επιστήμη της Βιομετρικής και στους λόγους που οδήγησαν στην ανάπτυξη μεθόδων και συστημάτων τα οποία μετράνε τα φυσικά χαρακτηριστικά ενός ατόμου. Στο δεύτερο κεφάλαιο αναλύονται οι αλγόριθμοι εξαγωγής της αγγειακής δομής από εικόνες δακτύλων ή παλάμης αλλά και του αμφιβληστροειδούς. Κάθε ένας από αυτούς τους αλγορίθμους βασίζεται στην εφαρμογή, επάνω στην εικόνα, διαφόρων φίλτρων και τεχνικών επεξεργασίας εικόνας. Στο τρίτο κεφάλαιο αναφέρονται οι βασικές εφαρμογές των συσκευών που εξάγουν την αγγειακή δομή. Τέτοιες συσκευές συναντάμε ως επί το πλείστον σε Συστήματα Ταυτοποίησης και Ασφαλείας. Ωστόσο χρησιμοποιούνται επίσης και για ιατρικούς σκοπούς σε διαγνωστικές και θεραπευτικές διαδικασίες. Στο τέταρτο κεφάλαιο αναλύονται οι βασικές τεχνικές φιλτραρίσματος και οι βασικές τεχνικές επεξεργασίας εικόνας, οι οποίες είναι και αυτές που μπορεί να εφαρμόσει ο χρήστης της Γραφικής Διεπαφής που υλοποιήθηκε. Στο πέμπτο και τελευταίο κεφάλαιο δίνεται ο σχεδιασμός και η ανάπτυξη του Γραφικού Περιβάλλοντος για την επεξεργασία εικόνας. Δίνεται η τελική μορφή της Γραφικής Διεπαφής και ο τρόπος λειτουργίας αυτής. / In this diploma work we studied, designed and created a Graphical User’s Interface in MATLAB environment through which a user can process a digital image tomography of a human finger. In the first chapter an introduction in Biometrics is made and we study the reasons that led in the development of methods and systems which recognize and measure the physical characteristics of a person. In the second chapter we analyze the algorithms which extract the vascular patterns from finger vein images and images of the palm dorsa and also from retinal images. Each one of them is based on the implementation, on the image, of several filters and image processing techniques. In the third chapter we mention the basic implementations of the devices which extract the vascular pattern. These devices are found mostly in Safety and Authentication Systems. However, they are also used for medical purposes in diagnostics and treatment’s procedures. In the fourth chapter we study the basic image filtering techniques and the basic image processing techniques. These techniques are also the options which are given to the user to implement on an image with the Graphical User’s Interface we created. In the fifth and last chapter the design and the creation of the Graphical User’s Interface for the image processing are given. Lastly, the final form of the Graphical User’s Interface and the way it can be used are given.

Automated human activity recognition has attracted increasing attention in the past decade. However, the application of machine learning and probabilistic methods for activity recognition problems has been studied only in the past couple of years. For the first time, this thesis explores the application of semi-supervised and active learning in activity recognition. We present a new and efficient semi-supervised training method for parameter estimation and feature selection in conditional random fields (CRFs),a probabilistic graphical model. In real-world applications such as activity recognition, unlabeled sensor traces are relatively easy to obtain whereas labeled examples are expensive and tedious to collect. Furthermore, the ability to automatically select a small subset of discriminatory features from a large pool can be advantageous in terms of computational speed as well as accuracy. We introduce the semi-supervised virtual evidence boosting (sVEB)algorithm for training CRFs — a semi-supervised extension to the recently developed virtual evidence boosting (VEB) method for feature selection and parameter learning. sVEB takes advantage of the unlabeled data via mini-mum entropy regularization. The objective function combines the unlabeled conditional entropy with labeled conditional pseudo-likelihood. The sVEB algorithm reduces the overall system cost as well as the human labeling cost required during training, which are both important considerations in building real world inference systems. Moreover, we propose an active learning algorithm for training CRFs is based on virtual evidence boosting and uses entropy measures. Active virtual evidence boosting (aVEB) queries the user for most informative examples, efficiently builds up labeled training examples and incorporates unlabeled data as in sVEB. aVEB not only reduces computational complexity of training CRFs as in sVEB, but also outputs more accurate classification results for the same fraction of labeled data. Ina set of experiments we illustrate that our algorithms, sVEB and aVEB, benefit from both the use of unlabeled data and automatic feature selection, and outperform other semi-supervised and active training approaches. The proposed methods could also be extended and employed for other classification problems in relational data.

This thesis contains the author’s work in and contributions to the field of Monte Carlo sampling for undirected graphical models, a class of statistical model commonly used in machine learning, computer vision, and spatial statistics; the aim is to be able to use the methodology and resultant samples to estimate integrals of functions of the variables in the model. Over the course of the study, three different but related methods were proposed and have appeared as research papers. The thesis consists of an introductory chapter discussing the models considered, the problems involved, and a general outline of Monte Carlo methods. The three subsequent chapters contain versions of the published work. The second chapter, which has appeared in (Hamze and de Freitas 2004), is a presentation of new MCMC algorithms for computing the posterior distributions and expectations of the unknown variables in undirected graphical models with regular structure. For demonstration purposes, we focus on Markov Random Fields (MRFs). By partitioning the MRFs into non-overlapping trees, it is possible to compute the posterior distribution of a particular tree exactly by conditioning on the remaining tree. These exact solutions allow us to construct efficient blocked and Rao-Blackwellised MCMC algorithms. We show empirically that tree sampling is considerably more efficient than other partitioned sampling schemes and the naive Gibbs sampler, even in cases where loopy belief propagation fails to converge. We prove that tree sampling exhibits lower variance than the naive Gibbs sampler and other naive partitioning schemes using the theoretical measure of maximal correlation. We also construct new information theory tools for comparing different MCMC schemes and show that, under these, tree sampling is more efficient. Although the work discussed in Chapter 2 exhibited promise on the class of graphs to which it was suited, there are many cases where limiting the topology is quite a handicap. The work in Chapter 3 was an exploration in an alternative methodology for approximating functions of variables representable as undirected graphical models of arbitrary connectivity with pairwise potentials, as well as for estimating the notoriously difficult partition function of the graph. The algorithm, published in (Hamze and de Freitas 2005), fits into the framework of sequential Monte Carlo methods rather than the more widely used MCMC, and relies on constructing a sequence of intermediate distributions which get closer to the desired one. While the idea of using “tempered” proposals is known, we construct a novel sequence of target distributions where, rather than dropping a global temperature parameter, we sequentially couple individual pairs of variables that are, initially, sampled exactly from a spanning treeof the variables. We present experimental results on inference and estimation of the partition function for sparse and densely-connected graphs. The final contribution of this thesis, presented in Chapter 4 and also in (Hamze and de Freitas 2007), emerged from some empirical observations that were made while trying to optimize the sequence of edges to add to a graph so as to guide the population of samples to the high-probability regions of the model. Most important among these observations was that while several heuristic approaches, discussed in Chapter 1, certainly yielded improvements over edge sequences consisting of random choices, strategies based on forcing the particles to take large, biased random walks in the state-space resulted in a more efficient exploration, particularly at low temperatures. This motivated a new Monte Carlo approach to treating complex discrete distributions. The algorithm is motivated by the N-Fold Way, which is an ingenious event-driven MCMC sampler that avoids rejection moves at any specific state. The N-Fold Way can however get “trapped” in cycles. We surmount this problem by modifying the sampling process to result in biased state-space paths of randomly chosen length. This alteration does introduce bias, but the bias is subsequently corrected with a carefully engineered importance sampler.

The modern programs must have a comfortable graphical user interface (GUI). Graphical interface is the environment for creation of graphical window with visual elements: buttons, edit text boxes, check boxes, list boxes, menu, toolbars etc., where a user chooses appropriate problem conditions, input data and sets proper options. For the development of such programs visual programming environments C++ Builder, Delphi, Visual Basic etc. were developed a decade ago. Recently user graphical interface technology was also implemented in the Computer Algebra Systems Maple (www.maplesoft.com), Mathematica (www.wolfram.com) and MatLab (www.mathworks.com). The present work investigates Maple graphical user interface creation facilities. It is possible to create GUI with Maplets package or in Maplet Builder design environment. This work examines these two methods with examples. The GUI creation facilities were tried investigating the curve fitting methods of Maple system. Some applied programs, using Maple Curve Fitting package, were created in Maple visual environment. The practical problem of speech sound formants variation curves fitting was also investigated. For that task some programs were created in the Maple visual environment.

This dissertation proposes a suite of novel approaches for solving multiagent decision and optimization problems based on the Collaborative Design Network (CDN), a framework for multiagent decision making. The framework itself is distributed, decision-theoretic and was originally proposed for multiagent component-centred design. This application is a novel use of the CDN, and demonstrate the generality of the CDN framework for general decision-theoretic planning. First, the framework is applied towards tackling a multiagent decision problem outside of collaborative design called multiagent expedition (MAE), a testbed problem which abstracts many of the features of real-world multiagent decision-making problems. We formally introduce MAE, and show it to be a subclass of a decentralized partially observable Markov Decision process (Dec-POMDP). We apply the CDN to the online MAE planning problem. We demonstrate that the CDN can plan in MAE with conditional optimality given a set of basic assumptions on the structure and organization of the agent team. We introduce a set of knowledge representational aspects to achieve conditionally optimal planning. We experimentally verify our approach on a series of benchmark problems created for this dissertation to test the various aspects of our CDN solution. We also investigate further methods for scalability and speedup in MAE. The concept of \emph{partial evaluation} (PE) is introduced, based on the assumption that an agent has an intended effect given an agent's action and considers all other effects unintended. This assumption is used to derive a bound for planning that partitions the set of joint plans into a set of fully evaluated and a set of partial evaluated plans. Plans which are partially evaluated can significantly speed up planning in the centralized case. PE is also applied to the CDN, to both public decisions between agents and private decisions local to an agent. We demonstrate that applying PE to public decisions in the CDN results in either intractable communication or suboptimal planning. When applied to private decisions, we show PE can still be very effective in decreasing planning runtime.

No description available.

The coarse mesh radiation transport (COMET) code uses response functions to solve the neutron transport equation. Most nuclear codes used today have a very steep learning curve; COMET is no exception. To ease the user's onus of learning how to create correctly formatted COMET input-files, a graphical user interface (GUI) was created. The GUI allows the user to select values for all the relevant variables while simultaneously minimizing the errors a typical new user would make. To this end, the GUI creates all of the input files required to run COMET. The GUI also provides a visualization tool that the user may use to check the problem geometry before running COMET. The GUI is also responsible for post-processing the COMET output for visualization with TecPlot. In addition to the GUI, multi-group cross section libraries were generated as part of the MHTGR-350 (Modular High Temperature Gas Reactor) benchmark problem under development at Georgia Tech. This project aims to couple COMET with a thermal hydraulics code to best model the true physics of the reactor design. In order for this goal to be actualized, six-group cross sections were generated over the operational temperature range of the MHTGR using the current coupling and collision probability code HELIOS.

This thesis contains the author’s work in and contributions to the field of Monte Carlo sampling for undirected graphical models, a class of statistical model commonly used in machine learning, computer vision, and spatial statistics; the aim is to be able to use the methodology and resultant samples to estimate integrals of functions of the variables in the model. Over the course of the study, three different but related methods were proposed and have appeared as research papers. The thesis consists of an introductory chapter discussing the models considered, the problems involved, and a general outline of Monte Carlo methods. The three subsequent chapters contain versions of the published work. The second chapter, which has appeared in (Hamze and de Freitas 2004), is a presentation of new MCMC algorithms for computing the posterior distributions and expectations of the unknown variables in undirected graphical models with regular structure. For demonstration purposes, we focus on Markov Random Fields (MRFs). By partitioning the MRFs into non-overlapping trees, it is possible to compute the posterior distribution of a particular tree exactly by conditioning on the remaining tree. These exact solutions allow us to construct efficient blocked and Rao-Blackwellised MCMC algorithms. We show empirically that tree sampling is considerably more efficient than other partitioned sampling schemes and the naive Gibbs sampler, even in cases where loopy belief propagation fails to converge. We prove that tree sampling exhibits lower variance than the naive Gibbs sampler and other naive partitioning schemes using the theoretical measure of maximal correlation. We also construct new information theory tools for comparing different MCMC schemes and show that, under these, tree sampling is more efficient. Although the work discussed in Chapter 2 exhibited promise on the class of graphs to which it was suited, there are many cases where limiting the topology is quite a handicap. The work in Chapter 3 was an exploration in an alternative methodology for approximating functions of variables representable as undirected graphical models of arbitrary connectivity with pairwise potentials, as well as for estimating the notoriously difficult partition function of the graph. The algorithm, published in (Hamze and de Freitas 2005), fits into the framework of sequential Monte Carlo methods rather than the more widely used MCMC, and relies on constructing a sequence of intermediate distributions which get closer to the desired one. While the idea of using “tempered” proposals is known, we construct a novel sequence of target distributions where, rather than dropping a global temperature parameter, we sequentially couple individual pairs of variables that are, initially, sampled exactly from a spanning treeof the variables. We present experimental results on inference and estimation of the partition function for sparse and densely-connected graphs. The final contribution of this thesis, presented in Chapter 4 and also in (Hamze and de Freitas 2007), emerged from some empirical observations that were made while trying to optimize the sequence of edges to add to a graph so as to guide the population of samples to the high-probability regions of the model. Most important among these observations was that while several heuristic approaches, discussed in Chapter 1, certainly yielded improvements over edge sequences consisting of random choices, strategies based on forcing the particles to take large, biased random walks in the state-space resulted in a more efficient exploration, particularly at low temperatures. This motivated a new Monte Carlo approach to treating complex discrete distributions. The algorithm is motivated by the N-Fold Way, which is an ingenious event-driven MCMC sampler that avoids rejection moves at any specific state. The N-Fold Way can however get “trapped” in cycles. We surmount this problem by modifying the sampling process to result in biased state-space paths of randomly chosen length. This alteration does introduce bias, but the bias is subsequently corrected with a carefully engineered importance sampler.

Over the past two decades graphical models have been widely used as powerful tools for compactly representing distributions. On the other hand, kernel methods have been used extensively to come up with rich representations. This thesis aims to combine graphical models with kernels to produce compact models with rich representational abilities. Graphical models are a powerful underlying formalism in machine learning. Their graph theoretic properties provide both an intuitive modular interface to model the interacting factors, and a data structure facilitating efficient learning and inference. The probabilistic nature ensures the global consistency of the whole framework, and allows convenient interface of models to data. Kernel methods, on the other hand, provide an effective means of representing rich classes of features for general objects, and at the same time allow efficient search for the optimal model. Recently, kernels have been used to characterize distributions by embedding them into high dimensional feature space. Interestingly, graphical models again decompose this characterization and lead to novel and direct ways of comparing distributions based on samples. Among the many uses of graphical models and kernels, this thesis is devoted to the following four areas: Conditional random fields for multi-agent reinforcement learning Conditional random fields (CRFs) are graphical models for modelling the probability of labels given the observations. They have traditionally been trained with using a set of observation and label pairs. Underlying all CRFs is the assumption that, conditioned on the training data, the label sequences of different training examples are independent and identically distributed (iid ). We extended the use of CRFs to a class of temporal learning algorithms, namely policy gradient reinforcement learning (RL). Now the labels are no longer iid. They are actions that update the environment and affect the next observation. From an RL point of view, CRFs provide a natural way to model joint actions in a decentralized Markov decision process. They define how agents can communicate with each other to choose the optimal joint action. We tested our framework on a synthetic network alignment problem, a distributed sensor network, and a road traffic control system. Using tree sampling by Hamze & de Freitas (2004) for inference, the RL methods employing CRFs clearly outperform those which do not model the proper joint policy. Bayesian online multi-label classification Gaussian density filtering (GDF) provides fast and effective inference for graphical models (Maybeck, 1982). Based on this natural online learner, we propose a Bayesian online multi-label classification (BOMC) framework which learns a probabilistic model of the linear classifier. The training labels are incorporated to update the posterior of the classifiers via a graphical model similar to TrueSkill (Herbrich et al., 2007), and inference is based on GDF with expectation propagation. Using samples from the posterior, we label the test data by maximizing the expected F-score. Our experiments on Reuters1-v2 dataset show that BOMC delivers significantly higher macro-averaged F-score than the state-of-the-art online maximum margin learners such as LaSVM (Bordes et al., 2005) and passive aggressive online learning (Crammer et al., 2006). The online nature of BOMC also allows us to effciently use a large amount of training data. Hilbert space embedment of distributions Graphical models are also an essential tool in kernel measures of independence for non-iid data. Traditional information theory often requires density estimation, which makes it unideal for statistical estimation. Motivated by the fact that distributions often appear in machine learning via expectations, we can characterize the distance between distributions in terms of distances between means, especially means in reproducing kernel Hilbert spaces which are called kernel embedment. Under this framework, the undirected graphical models further allow us to factorize the kernel embedment onto cliques, which yields efficient measures of independence for non-iid data (Zhang et al., 2009). We show the effectiveness of this framework for ICA and sequence segmentation, and a number of further applications and research questions are identified. Optimization in maximum margin models for structured data Maximum margin estimation for structured data, e.g. (Taskar et al., 2004), is an important task in machine learning where graphical models also play a key role. They are special cases of regularized risk minimization, for which bundle methods (BMRM, Teo et al., 2007) and the closely related SVMStruct (Tsochantaridis et al., 2005) are state-of-the-art general purpose solvers. Smola et al. (2007b) proved that BMRM requires O(1/έ) iterations to converge to an έ accurate solution, and we further show that this rate hits the lower bound. By utilizing the structure of the objective function, we devised an algorithm for the structured loss which converges to an έ accurate solution in O(1/√έ) iterations. This algorithm originates from Nesterov's optimal first order methods (Nesterov, 2003, 2005b).