This thesis proposes and tests a recursive, adpative, and computationally inexpensive method for partitioning real-number spaces. When tested for proof-of-concept on both one- and two- dimensional classification and control problems, the Fern algorithm was found to work well in one dimension, moderately well for two-dimensional classification, and not at all for two-dimensional control. Testing ferns as pure discretizers - which would involve a secondary discrete learner - has been left to future work. / text

Abstract Cardiovascular disease is one of the leading causes of death in Canada. Atherosclerosis is considered the primary cause for cardiovascular disease. Optical coherence tomography (OCT) provides a means to minimally invasive imaging and assessment of textural features of atherosclerotic plaque. However, detecting atherosclerotic plaque by visual inspection from Optical Coherence Tomography (OCT) images is usually difficult. Therefore we developed unsupervised segmentation algorithms to automatically detect atherosclerosis plaque from OCT images. We used three different clustering methods to identify atherosclerotic plaque automatically from OCT images. Our method involves data preprocessing of raw OCT images, feature selection and texture feature extraction using the Spatial Gray Level Dependence Matrix method (SGLDM), and the application of three different clustering techniques: K-means, Fuzzy C-means and Gustafson-Kessel algorithms to segment the plaque regions from OCT images and to map the cluster regions (background, vascular tissue, OCT degraded signal region and Atherosclerosis plaque) from the feature-space back to the original preprocessed OCT image. We validated our results by comparing our segmented OCT images with actual photographic images of vascular tissue with plaque. / October 2015

The objective of this research project was to develop a methodology to establish the potential of automated sorting for a minerals application. Such methodologies, have been developed for testwork in many established mineral processing disciplines. These techniques ensure that data is reproducible and that testing can be undertaken in a quick and efficient manner. Due to the relatively recent development of automated sorters as a mineral processing technique, such guidelines have yet to be established. The methodology developed was applied to two practical applications including the separation of a Ni/Cu sulphide ore. This experimentation also highlighted the advantages of multi-sensor sorting and illustrated a means by which sorters can be used as multi-output machines; generating a number of tailored concentrates for down-stream processing. This is in contrast to the traditional view of sorters as a simple binary, concentrate/waste pre-concentration technique. A further key result of the research was the emulation of expert-based training using unsupervised clustering techniques and neural networks for colour quantisation. These techniques add flexibility and value to sorters in the minerals industry as they do not require a trained expert and so allow machines to be optimised by mine operators as conditions vary. The techniques also have an advantage as they complete the task of colour quantisation in a fraction of the time taken for an expert and so lend themselves well to the quick and efficient determination of automated sorting for a minerals application. Future research should focus on the advancement and application of neural networks to colour quantisation in conjunction with tradition training methods Further to this research should concentrate on practical applications utilising a multi-sensor, multi-output approach to automated sorting.

The subject matter of this thesis is the problem of learning to discover grammatical structure from raw text alone, without access to explicit instruction or annotation -- in particular, by a computer or computational process -- in other words, unsupervised parser induction, or simply, unsupervised parsing. This work presents a method for raw text unsupervised parsing that is simple, but nevertheless achieves state-of-the-art results on treebank-based direct evaluation. The approach to unsupervised parsing presented in this dissertation adopts a different way to constrain learned models than has been deployed in previous work. Specifically, I focus on a sub-task of full unsupervised partial parsing called unsupervised partial parsing. In essence, the strategy is to learn to segment a string of tokens into a set of non-overlapping constituents or chunks which may be one or more tokens in length. This strategy has a number of advantages: it is fast and scalable, based on well-understood and extensible natural language processing techniques, and it produces predictions about human language structure which are useful for human language technologies. The models developed for unsupervised partial parsing recover base noun phrases and local constituent structure with high accuracy compared to strong baselines. Finally, these models may be applied in a cascaded fashion for the prediction of full constituent trees: first segmenting a string of tokens into local phrases, then re-segmenting to predict higher-level constituent structure. This simple strategy leads to an unsupervised parsing model which produces state-of-the-art results for constituent parsing of English, German and Chinese. This thesis presents, evaluates and explores these models and strategies. / text

Unsupervised learning has become more and more important due to the recent explosion of data. Clustering, a key topic in unsupervised learning, is a well-studied task arising in many applications ranging from computer vision to computational biology to the social sciences. This thesis is a collection of work exploring two modern aspects of clustering: stability and scalability. In the first part, we study clustering under a stability property called perturbation resilience. As an alternative approach to worst case analysis, this novel theoretical framework aims at understanding the complexity of clustering instances that satisfy natural stability assumptions. In particular, we show how to correctly cluster instances whose optimal solutions are resilient to small multiplicative perturbations on the distances between data points, significantly improving existing guarantees. We further propose a generalized property that allows small changes in the optimal solutions after perturbations, and provide the first known positive results in this more challenging setting. In the second part, we study the problem of clustering large scale data distributed across nodes which communicate over the edges of a connected graph. We provide algorithms with small communication cost and provable guarantees on the clustering quality. We also propose algorithms for distributed principal component analysis, which can be used to reduce the communication cost of clustering high dimensional data while merely comprising the clustering quality. In the third part, we study community detection, the modern extension of clustering to network data. We propose a theoretical model of communities that are stable in the presence of noisy nodes in the network, and design an algorithm that provably detects all such communities. We also provide a local algorithm for large scale networks, whose running time depends on the sizes of the output communities but not that of the entire network.

Many high-dimensional time-varying signals can be modeled as a sequence of noisy nonlinear observations of a low-dimensional dynamical process. Given high-dimensional observations and a distribution describing the dynamical process, we present a computationally inexpensive approximate algorithm for estimating the inverse of this mapping. Once this mapping is learned, we can invert it to construct a generative model for the signals. Our algorithm can be thought of as learning a manifold of images by taking into account the dynamics underlying the low-dimensional representation of these images. It also serves as a nonlinear system identification procedure that estimates the inverse of the observation function in nonlinear dynamic system. Our algorithm reduces to a generalized eigenvalue problem, so it does not suffer from the computational or local minimum issues traditionally associated with nonlinear system identification, allowing us to apply it to the problem of learning generative models for video sequences.

Current computer vision techniques can effectively monitor gross activities in sparse environments. Unfortunately, visual stimulus is often not sufficient for reliably discriminating between many types of activity. In many cases where the visual information required for a particular task is extremely subtle or non-existent, there is often audio stimulus that is extremely salient for a particular classification or anomaly detection task. Unfortunately unlike visual events, independent sounds are often very ambiguous and not sufficient to define useful events themselves. Without an effective method of learning causally-linked temporal sequences of sound events that are coupled to the visual events, these sound events are generally only useful for independent anomalous sounds detection, e.g., detecting a gunshot or breaking glass. This paper outlines a method for automatically detecting a set of audio events and visual events in a particular environment, for determining statistical anomalies, for automatically clustering these detected events into meaningful clusters, and for learning salient temporal relationships between the audio and visual events. This results in a compact description of the different types of compound audio-visual events in an environment.

No description available.

The final thesis summarizes in its theoretical part basic knowledge of machine learning algorithms that involves supervised, semi-supervised, and unsupervised learning. Experiments with textual data in natural spoken language involving different machine learning methods and parameterization are carried out in its practical part. Conclusions made in the thesis may be of use to individuals that are at least slightly interested in this domain.

Representing and handling high-dimensional data has been increasingly ubiquitous in many real world-applications, such as computer vision, machine learning, and data mining. High-dimensional data usually have intrinsic low-dimensional structures, which are suitable for subsequent data processing. As a consequent, it has been a common demand to find low-dimensional data representations in many machine learning and data mining problems. Factorization methods have been impressive in recovering intrinsic low-dimensional structures of the data. When seeking low-dimensional representation of the data, traditional methods mainly face two challenges: 1) how to discover the most variational features/information from the data; 2) how to measure accurate nonlinear relationships of the data. As a solution to these challenges, traditional methods usually make use of a two-step approach by performing feature selection and manifold construction followed by further data processing, which omits the dependence between these learning tasks and produce inaccurate data representation. To resolve these problems, we propose to integrate feature learning and graph learning with factorization model, which allows the goals of learning features, constructing manifold, and seeking new data representation to mutually enhance and lead to powerful data representation capability. Moreover, it has been increasingly common that 2-dimensional (2D) data often have high dimensions of features, where each example of 2D data is a matrix with its elements being features. For such data, traditional data usually convert them to 1-dimensional vectorial data before data processing, which severely damages inherent structures of such data. We propose to directly use 2D data for seeking new representation, which enables the model to preserve inherent 2D structures of the data. We propose to seek projection directions to find the subspaces, in which spatial information is maximumly preserved. Also, manifold and new data representation are learned in these subspaces, such that the manifold are clean and the new representation is discriminative. Consequently, seeking projections, learning manifold and constructing new representation mutually enhance and lead to powerful data representation technique.