A wide range of statistical and machine learning problems involve learning one or multiple latent functions, or properties thereof, from datasets. Examples include regression, classification, principal component analysis, optimisation, learning intensity functions of point processes and reinforcement learning to name but a few. For all these problems, positive semi-definite kernels (or simply kernels) provide a powerful tool for postulating flexible nonparametric hypothesis spaces over functions. Despite recent work on such kernel methods, parametric alternatives, such as deep neural networks, have been at the core of most artificial intelligence breakthroughs in recent years. In this thesis, both theoretical and methodological foundations are presented for constructing fully automated, scalable, and general-purpose kernel machines that perform very well over a wide range of input dimensions and sample sizes. This thesis aims to contribute towards bridging the gap between kernel methods and deep learning and to propose methods that have the advantage over deep learning in performing well on both small and large scale problems. In Part I we provide a gentle introduction to kernel methods, review recent work, identify remaining gaps and outline our contributions. In Part II we develop flexible and scalable Bayesian kernel methods in order to address gaps in methods capable of dealing with the special case of datasets exhibiting locally homogeneous patterns. We begin with two motivating applications. First we consider inferring the intensity function of an inhomogeneous point process in Chapter 2. This application is used to illustrate that often, by carefully adding some mild asymmetry in the dependency structure in Bayesian kernel methods, one may considerably scale-up inference while improving flexibility and accuracy. In Chapter 3 we propose a scalable scheme for online forecasting of time series and fully-online learning of related model parameters, under a kernel-based generative model that is provably sufficiently flexible. This application illustrates that, for one-dimensional input spaces, restricting the degree of differentiability of the latent function of interest may considerably speed-up inference without resorting to approximations and without any adverse effect on flexibility or accuracy. Chapter 4 generalizes these approaches and proposes a novel class of stochastic processes we refer to as string Gaussian processes (string GPs) that, when used as functional prior in a Bayesian nonparametric framework, allow for inference in linear time complexity and linear memory requirement, without resorting to approximations. More importantly, the corresponding inference scheme, which we derive in Chapter 5, also allows flexible learning of locally homogeneous patterns and automated learning of model complexity - that is automated learning of whether there are local patterns in the data in the first place, how much local patterns are present, and where they are located. In Part III we provide a broader discussion covering all types of patterns (homogeneous, locally homogeneous or heterogeneous patterns) and both Bayesian or frequentist kernel methods. In Chapter 6 we begin by discussing what properties a family of kernels should possess to enable fully automated kernel methods that are applicable to any type of datasets. In this chapter, we discuss a novel mathematical formalism for the notion of ‘general-purpose' families of kernels, and we argue that existing families of kernels are not general-purpose. In Chapter 7 we derive weak sufficient conditions for families of kernels to be general-purpose, and we exhibit tractable such families that enjoy a suitable parametrisation, that we refer to as generalized spectral kernels (GSKs). In Chapter 8 we provide a scalable inference scheme for automated kernel learning using general-purpose families of kernels. The proposed inference scheme scales linearly with the sample size and enables automated learning of nonstationarity and model complexity from the data, in virtually any kernel method. Finally, we conclude with a discussion in Chapter 9 where we show that deep learning can be regarded as a particular type of kernel learning method, and we discuss possible extensions in Chapter 10.

This thesis is concerned with sequence logic as found in industrial control systems, with the focus being on process and manufacturing control systems. At its core is the assertion that there is a need for a better approach to the development of industrial sequence logic to satisfy the life-cycle requirements, and that many of the ingredients required to deliver such an approach are now available. The needs are discussed by considering the business case for automation and deficiencies with traditional approaches. A set of requirements is then derived for an integrated development environment to address the business needs throughout the control system life-cycle. The strengths and weaknesses of relevant control system technology and standards are reviewed and their bias towards implementation described. Mathematical models, graphical methods and software tools are then assessed with respect to the requirements for an integrated development environment. A solution to the requirements, called Synect is then introduced. Synect combines a methodology using familiar graphical notations with Petri net modelling supported by a set of software tools. Its key features are justified with reference to the requirements. A set of case studies forms the basis of an evaluation against business needs by comparing the Synect methodology with current approaches. The industrial relevance and exploitation are then briefly described. The thesis ends with a review of the key conclusions along with contributions to knowledge and suggestions for further research.

Structured output prediction is a machine learning tasks in which an input object is not just assigned a single class, as in classification, but multiple, interdependent labels. This means that the presence or value of a given label affects the other labels, for instance in text labelling problems, where output labels are applied to each word, and their interdependencies must be modelled. Non-parametric Bayesian (NPB) techniques are probabilistic modelling techniques which have the interesting property of allowing model capacity to grow, in a controllable way, with data complexity, while maintaining the advantages of Bayesian modelling. In this thesis, we develop NPB algorithms to solve structured output problems. We first study a map-reduce implementation of a stochastic inference method designed for the infinite hidden Markov model, applied to a computational linguistics task, part-of-speech tagging. We show that mainstream map-reduce frameworks do not easily support highly iterative algorithms. The main contribution of this thesis consists in a conceptually novel discriminative model, GPstruct. It is motivated by labelling tasks, and combines attractive properties of conditional random fields (CRF), structured support vector machines, and Gaussian process (GP) classifiers. In probabilistic terms, GPstruct combines a CRF likelihood with a GP prior on factors; it can also be described as a Bayesian kernelized CRF. To train this model, we develop a Markov chain Monte Carlo algorithm based on elliptical slice sampling and investigate its properties. We then validate it on real data experiments, and explore two topologies: sequence output with text labelling tasks, and grid output with semantic segmentation of images. The latter case poses scalability issues, which are addressed using likelihood approximations and an ensemble method which allows distributed inference and prediction. The experimental validation demonstrates: (a) the model is flexible and its constituent parts are modular and easy to engineer; (b) predictive performance and, most crucially, the probabilistic calibration of predictions are better than or equal to that of competitor models, and (c) model hyperparameters can be learnt from data.

Prototyping is an essential step in the manufacture of many objects, both consumer and industrial. A fundamental step in this process is the definition of the three dimensional form of the object shape; for example, a designer's models created in clay or wood. A three dimensional vision system (range sensor) offers the advantage of speed in defining shapes compared to traditional tactile sensing. In this thesis, the viability of using range sensors is demonstrated by developing a rapid prototyping system comprised of a laser-based range sensor and software that creates a computer model of the object. One particularly important application of the computer model is for the generation of a control program, or toolpath, for a computer-numerically-controlled (CNC) machine tool. This is an important application in mold and die manufacture and mold manufacture for automobile components from full scale models. The computer model can also be incorporated into computer aided design and analysis programs. The most suitable vision system, for rapid prototyping applications, has been selected from a group of available sensors and integrated with a coordinate measuring machine that acts as a translation system. The range data produced have been utilised in a multi-patch surface modelling approach in order to model objects where many types of surface patches, such as quadric and free form, are blended together on one object. This technique has been demonstrated to provide accurate and smooth surface reconstructions that are suitable for generating CNC toolpaths. The viability of machining from multiple surface patch models has been demonstrated and, in addition, a new technique for the machining of free form surfaces developed. An alternative method for fully defining complex three dimensional shapes employing a rotary sensing of the object is also presented that permits the efficient generation of CNC machine toolpaths. / Graduate

CCTV cameras produce a large amount of video surveillance data per day, and analysing them require the use of significant computing resources that often need to be scalable. The emergence of the Hadoop distributed processing framework has had a significant impact on various data intensive applications as the distributed computed based processing enables an increase of the processing capability of applications it serves. Hadoop is an open source implementation of the MapReduce programming model. It automates the operation of creating tasks for each function, distribute data, parallelize executions and handles machine failures that reliefs users from the complexity of having to manage the underlying processing and only focus on building their application. It is noted that in a practical deployment the challenge of Hadoop based architecture is that it requires several scalable machines for effective processing, which in turn adds hardware investment cost to the infrastructure. Although using a cloud infrastructure offers scalable and elastic utilization of resources where users can scale up or scale down the number of Virtual Machines (VM) upon requirements, a user such as a CCTV system operator intending to use a public cloud would aspire to know what cloud resources (i.e. number of VMs) need to be deployed so that the processing can be done in the fastest (or within a known time constraint) and the most cost effective manner. Often such resources will also have to satisfy practical, procedural and legal requirements. The capability to model a distributed processing architecture where the resource requirements can be effectively and optimally predicted will thus be a useful tool, if available. In literature there is no clear and comprehensive modelling framework that provides proactive resource allocation mechanisms to satisfy a user's target requirements, especially for a processing intensive application such as video analytic. In this thesis, with the hope of closing the above research gap, novel research is first initiated by understanding the current legal practices and requirements of implementing video surveillance system within a distributed processing and data storage environment, since the legal validity of data gathered or processed within such a system is vital for a distributed system's applicability in such domains. Subsequently the thesis presents a comprehensive framework for the performance ii modelling and optimization of resource allocation in deploying a scalable distributed video analytic application in a Hadoop based framework, running on virtualized cluster of machines. The proposed modelling framework investigates the use of several machine learning algorithms such as, decision trees (M5P, RepTree), Linear Regression, Multi Layer Perceptron(MLP) and the Ensemble Classifier Bagging model, to model and predict the execution time of video analytic jobs, based on infrastructure level as well as job level parameters. Further in order to propose a novel framework for the allocate resources under constraints to obtain optimal performance in terms of job execution time, we propose a Genetic Algorithms (GAs) based optimization technique. Experimental results are provided to demonstrate the proposed framework's capability to successfully predict the job execution time of a given video analytic task based on infrastructure and input data related parameters and its ability determine the minimum job execution time, given constraints of these parameters. Given the above, the thesis contributes to the state-of-art in distributed video analytics, design, implementation, performance analysis and optimisation.

Master of Science / Department of Computer Science / William H. Hsu / This thesis addresses the tasks of detecting vegetation and classifying plants into target crops and weeds using combinations of machine learning and pattern recognition algorithms and models. Solutions to these problems have many useful applications in precision agriculture, such as estimating the yield of a target crop or identifying weeds to help automate the selective application of weedicides and thereby reducing cost and pollution. The novel contribution of this work includes development and application of image processing and computer vision techniques to create training data with minimal human intervention, thus saving substantial human time and effort. All of the data used in this work was collected from corn fields and is in the RGB format. As part of this thesis, I first discuss several steps that are part of a general methodology and data science pipeline for these tasks, such as: vegetation detection, feature engineering, crop row detection, training data generation, training, and testing. Next, I develop software components for segmentation and classification subtasks based on extant image processing and machine learning algorithms. I then present a comparison of different classifier models developed through this process using their Receiver Operating Characteristic (ROC) curves. The difference in models lies in the way they are trained - locally or globally. I also investigate the effect of the altitude at which data is collected on the performance of classifiers. Scikit-learn, a Python library for machine learning, is used to train decision trees and other classification learning models. Finally, I compare the precision, recall, and accuracy attained by segmenting (recognizing the boundary of) plants using the excess green index (ExG) with that of a learned Gaussian mixture model. I performed all image processing tasks using OpenCV, an open source computer vision library.

No description available.

Abstract The aim of this technologically oriented study was to develop an evaluation method for post-completion audits of investment projects in paper production lines. The development work was based on the constructive research approach. The objectives of the method were practical applicability, a comprehensive framework, and measures of project performance. The evaluation method developed here is based on an adapted Malcolm Baldrige self-assessment framework, which has been embedded with three evaluation perspectives: project Targets (T), Risk management (R), and company Strategy (S). The compositions of these perspectives i.e. TRS perspectives, serve as the fundamental basis for a value added project. The evaluation criteria have been designed to fulfil the requirements of the paper industry and demanding projects. Scoring guidelines are used as a basis for evaluation results. Furthermore, supportive evaluation tools were developed to improve the accuracy and comparability of evaluation results. Evaluation is conducted by a group which consists of qualified tutors, project experts and facilitators. The latter are used especially to increase objectivity. The method was tested with two cases studies which were applied in a greenfield paper machine project. The first case study focused on technology choices. The second case study was a full-scale study of the whole project scope. The evaluation results were relatively accurate, and feedback results were particularly positive. Usage of the TRS perspectives can produce information which benefits decision-making. The method is capable of measuring both technical facts and subjective opinions. Moreover, the method is applicable in practice and can improve the investment process in general.

We address the challenging task of computational natural language inference, by which we mean bridging two or more natural language texts while also providing an explanation of how they are connected. In the context of question answering (i.e., finding short answers to natural language questions), this inference connects the question with its answer and we learn to approximate this inference with machine learning. In particular, here we present four approaches to question answering, each of which shows a significant improvement in performance over baseline methods. In our first approach, we make use of the underlying discourse structure inherent in free text (i.e. whether the text contains an explanation, elaboration, contrast, etc.) in order to increase the amount of training data for (and subsequently the performance of) a monolingual alignment model. In our second work, we propose a framework for training customized lexical semantics models such that each one represents a single semantic relation. We use causality as a use case, and demonstrate that our customized model is able to both identify causal relations as well as significantly improve our ability to answer causal questions. We then propose two approaches that seek to answer questions by learning to rank human-readable justifications for the answers, such that the model selects the answer with the best justification. The first uses a graph-structured representation of the background knowledge and performs information aggregation to construct multi-sentence justifications. The second reduces pre-processing costs by limiting itself to a single sentence and using a neural network to learn a latent representation of the background knowledge. For each of these, we show that in addition to significant improvement in correctly answering questions, we also outperform a strong baseline in terms of the quality of the answer justification given.

Master of Science / Department of Computing and Information Sciences / David A. Gustafson / An interest point detection scheme is presented that is comparable in quality to existing methods, but can be performed much faster. The detection is based on a straightforward color analysis at a coarse granularity. A 3x3 grid of squares is centered on the candidate point, so that the candidate point corresponds to the middle square. If the color of the center region is inhomogeneous with all of the surrounding regions, the point is labeled as interesting. A point will also be labeled as interesting if a minority of the surrounding squares are homogeneous, and arranged in an appropriate pattern. Testing confirms that this detection scheme is much faster than the state-of-the-art. It is also repeatable, even under different viewing conditions. The detector is robust with respect to changes in viewpoint, lighting, zoom, and to a certain extent, rotation.