Doctor of Philosophy / Department of Computer Science / William H. Hsu / The contribution of the proposed dissertation is the application of affective intelligence in human-developed spaces where people live, work, and recreate daily, also known as built environments. Built environments have been known to influence and impact individual affective responses. The implications of built environments on human well-being and mental health necessitate the need to develop new metrics to measure and detect how humans respond subjectively in built environments. Detection of arousal in built environments given biometric data and environmental characteristics via a machine learning-centric approach provides a novel and new capability to measure human responses to built environments. Work was also conducted on experimental design methodologies for multiple sensor fusion and detection of affect in built environments. These contributions include exploring new methodologies in applying supervised machine learning algorithms, such as logistic regression, random forests, and artificial neural networks, in the detection of arousal in built environments. Results have shown a machine learning approach can not only be used to detect arousal in built environments but also for the construction of novel explanatory models of the data.

Many applications collect a large number of time series, for example, temperature continuously monitored by weather stations across the US or neural activity recorded by an array of electrical probes. These data are often referred to as unstructured. A first task in their analytics is often to derive a low dimensional representation { a graph or discrete manifold { that describes the inter relations among the time series and their intrarelations across time. In general, the underlying graphs can be directed and weighted, possibly capturing the strengths of causal relations, not just the binary existence of reciprocal correlations. Furthermore, the processes generating the data may be non-linear and observed in the presence of unmodeled phenomena or unmeasured agents in a complex networked system. Finally, the networks describing the processes may themselves vary through time. In many scenarios, there may be good reasons to believe that the graphs are only able to vary as linear combinations of a set of \principal graphs" that are fundamental to the system. We would then be able to characterize each principal network individually to make sense of the ensemble and analyze the behaviors of the interacting entities. This thesis acts as a roadmap of computationally tractable approaches for learning graphs that provide structure to data. It culminates in a framework that addresses these challenges when estimating time-varying graphs from collections of time series. Analyses are carried out to justify the various models proposed along the way and to characterize their performance. Experiments are performed on synthetic and real datasets to highlight their effectiveness and to illustrate their limitations.

Regulation of protein activity is essential for normal cell functionality. Many proteins are regulated allosterically, that is, with spatial gaps between stimulation and active sites. Biological stimuli that regulate proteins allosterically include, for example, ions and small molecules, post-translational modifications, and intensive state-variables like temperature and pH. These effectors can not only switch activities on-and-off, but also fine-tune activities. Understanding the underpinnings of allostery, that is, how signals are propagated between distant sites, and how transmitted signals manifest themselves into regulation of protein activity, has been one of the central foci of biology for over 50 years. Today, the importance of such studies goes beyond basic pedagogical interests as bioengineers seek design features to control protein function for myriad purposes, including design of nano-biosensors, drug delivery vehicles, synthetic cells and organic-synthetic interfaces. The current phenomenological view of allostery is that signaling and activity control occur via effector-induced changes in protein conformational ensembles. If the structures of two states of a protein differ from each other significantly, then thermal fluctuations can be neglected and an atomically detailed model of regulation can be constructed in terms of how their minimum-energy structures differ between states. However, when the minimum-energy structures of states differ from each other only marginally and the difference is comparable to thermal fluctuations, then a mechanistic model cannot be constructed solely on the basis of differences in protein structure. Understanding the mechanism of dynamic allostery requires not only assessment of high-dimensional conformational ensembles of the various individual states, including inactive, transition and active states, but also relationships between them. This challenge faces many diverse protein families, including G-protein coupled receptors, immune cell receptors, heat shock proteins, nuclear transcription factors and viral attachment proteins, whose mechanisms, despite numerous studies, remain poorly understood. This dissertation deals with the development of new methods that significantly boost the applicability of molecular simulation techniques to probe dynamic allostery in these proteins. Specifically, it deals with two different methods, one to obtain quantitative estimates for subtle differences between conformational ensembles, and the other to relate conformational ensemble differences to allosteric signal communication. Both methods are enabled by a new application of the mathematical framework of machine learning. These methods are applied to (a) identify specific effects of employed force fields on conformational ensembles, (b) compare multiple ensembles against each other for determination of common signaling pathways induced by different effectors, (c) identify the effects of point mutations on conformational ensemble shifts in proteins, and (d) understand the mechanism of dynamic allostery in a PDZ domain. These diverse applications essentially demonstrate the generality of the developed approaches, and specifically set the foundation for future studies on PDZ domains and viral attachment proteins.

The notion of salience in the discourse analysis models how the activation of referred objects evolves in the flow of text. The salience algorithm was already defined and tested briefly in an earlier research, we present a reproduction of its results in a larger scale using data from the Prague Dependency Treebank 3.0. The results are then collected into an accessible shape and analyzed both in their visual and quantitative form in the context of the two main resources of the salience - coreference relations and topic-focus articulation. Furthermore, a possibility of modeling the salience degree by a machine learning algorithm (decision trees and random forest) is examined. Finally, attempts are made with using the salience information in the machine learning NLP task of document clustering visualization. Powered by TCPDF (www.tcpdf.org)

Vocal cord vibration is the source of voiced phonemes in speech. Voice quality depends on the nature of this vibration. Vocal cords can be damaged by infection, neck or chest injury, tumours and more serious diseases such as laryngeal cancer. This kind of physical damage can cause loss of voice quality. To support the diagnosis of such conditions and also to monitor the effect of any treatment, voice quality assessment is required. Traditionally, this is done ‘subjectively’ by Speech and Language Therapists (SLTs) who, in Europe, use a well-known assessment approach called ‘GRBAS’. GRBAS is an acronym for a five dimensional scale of measurements of voice properties. The scale was originally devised and recommended by the Japanese Society of Logopeadics and Phoniatrics and several European research publications. The proper- ties are ‘Grade’, ‘Roughness’, ‘Breathiness’, ‘Asthenia’ and ‘Strain’. An SLT listens to and assesses a person’s voice while the person performs specific vocal maneuvers. The SLT is then required to record a discrete score for the voice quality in range of 0 to 3 for each GRBAS component. In requiring the services of trained SLTs, this subjective assessment makes the traditional GRBAS procedure expensive and time-consuming to administer. This thesis considers the possibility of using computer programs to perform objective assessments of voice quality conforming to the GRBAS scale. To do this, Digital Signal Processing (DSP) algorithms are required for measuring voice features that may indicate voice abnormality. The computer must be trained to convert DSP measurements to GRBAS scores and a ‘machine learning’ approach has been adopted to achieve this. This research was made possible by the development, by Manchester Royal Infirmary (MRI) Hospital Trust, of a ‘speech database’ with the participation of clinicians, SLT’s, patients and controls. The participation of five SLTs scorers allowed norms to be established for GRBAS scoring which provided ‘reference’ data for the machine learning approach.
To support the scoring procedure carried out at MRI, a software package, referred to as GRBAS Presentation and Scoring Package (GPSP), was developed for presenting voice recordings to each of the SLTs and recording their GRBAS scores. A means of assessing intra-scorer consistency was devised and built into this system. Also, the assessment of inter-scorer consistency was advanced by the invention of a new form of the ‘Fleiss Kappa’ which is applicable to ordinal as well as categorical scoring. The means of taking these assessments of scorer consistency into account when producing ‘reference’ GRBAS scores are presented in this thesis. Such reference scores are required for training the machine learning algorithms. The DSP algorithms required for feature measurements are generally well known and available as published or commercial software packages. However, an appraisal of these algorithms and the development of some DSP ‘thesis software’ was found to be necessary. Two ‘machine learning’ regression models have been developed for map- ping the measured voice features to GRBAS scores. These are K Nearest Neighbor Regression (KNNR) and Multiple Linear Regression (MLR). Our research is based on sets of features, sets of data and prediction models that are different from the approaches in the current literature. The performance of the computerised system is evaluated against reference scores using a Normalised Root Mean Squared Error (NRMSE) measure. The performances of MLR and KNNR for objective prediction of GRBAS scores are compared and analysed ‘with feature selection’ and ‘without feature selection’. It was found that MLR with feature selection was better than MLR without feature selection and KNNR with and without feature selection, for all five GRBAS components. It was also found that MLR with feature selection gives scores for ‘Asthenia’ and ‘Strain’ which are closer to the reference scores than the scores given by all five individual SLT scorers. The best objective score for ‘Roughness’ was closer than the scores given by two SLTs, roughly equal to the score of one SLT and worse than the other two SLT scores. The best objective scores for ‘Breathiness’ and ‘Grade’ were further from the reference scores than the scores produced by all five SLT scorers. However, the worst ‘MLR with feature selection’ result has normalised RMS error which is only about 3% worse than the worst SLT scoring. The results obtained indicate that objective GRBAS measurements have the potential for further development towards a commercial product that may at least be useful in augmenting the subjective assessments of SLT scorers.

During flights with manned or unmanned aircraft, continuous recording can result in avery high number of images to analyze and evaluate. To simplify image analysis and tominimize data link usage, appropriate images should be suggested for transfer and furtheranalysis. This thesis investigates features used for selection of images worthy of furtheranalysis using machine learning. The selection is done based on the criteria of havinggood quality, salient content and being unique compared to the other selected images.The investigation is approached by implementing two binary classifications, one regardingcontent and one regarding quality. The classifications are made using support vectormachines. For each of the classifications three feature extraction methods are performedand the results are compared against each other. The feature extraction methods used arehistograms of oriented gradients, features from the discrete cosine transform domain andfeatures extracted from a pre-trained convolutional neural network. The images classifiedas both good and salient are then clustered based on similarity measures retrieved usingcolor coherence vectors. One image from each cluster is retrieved and those are the resultingimages from the image selection. The performance of the selection is evaluated usingthe measures precision, recall and accuracy. The investigation showed that using featuresextracted from the discrete cosine transform provided the best results for the quality classification.For the content classification, features extracted from a convolutional neuralnetwork provided the best results. The similarity retrieval showed to be the weakest partand the entire system together provides an average accuracy of 83.99%.

Title: Machine Learning for Credit Scoring Author: Elena Myazina Department / Institute: Department of Theoretical Computer Science and Mathematical Logic Supervisor of the master thesis: Mgr. Martin Pilát, Ph.D, Department of Theoretical Computer Science and Mathematical Logic Abstract: Credit scoring is a technique used by banks to evaluate their clients who ask for different types of loan. Its goal is to predict, whether a given client will pay their loan or not. Traditionally, mathematical models based on logistic regression are used for this task. In this thesis, we approach the problem of credit scoring from a machine learning point of view. We investigate several machine learning methods (including neural networks, random forests, support vector machines and other), and evaluate their performance for the credit scoring task on three publicly available datasets.. Keywords: machine learning, credit scoring,logistic regression, neural networks, random forest

Frequent Itemset Mining, the process of finding frequently co-occurring sets of items in a dataset, has been at the core of the field of data mining for the past 25 years. During this time the datasets have grown much faster than the algorithms capacity to process them. Great progress was made at optimizing this task on a single computer however, despite years of research, very little progress has been made on parallelizing this task. FPGrowth based algorithms have proven notoriously difficult to parallelize and Apriori has largely fallen out of favor with the research community. In this thesis we introduce a parallel, Apriori based, Frequent Itemset Mining algo- rithm capable of distributing computation across large commodity clusters. Our case study demonstrates that our algorithm can efficiently scale to hundreds of cores, on a standard Hadoop MapReduce cluster, and can improve executions times by at least an order of magnitude at the lowest support levels. / Graduate / 0984 / 0800 / nclark@uvic.ca

Applications for Internet-enabled devices use machine learning to process captured data to make intelligent decisions or provide information to users. Typically, the computation to process the data is executed in cloud-based backends. The devices are used for sensing data, offloading it to the cloud, receiving responses and acting upon them. However, this approach leads to high end-to-end latency due to communication over the Internet. This dissertation proposes reducing this response time by minimizing offloading, and pushing computation close to the source of the data, i.e. to edge servers and devices themselves. To adapt to the resource constrained environment at the edge, it presents an approach that leverages spatiotemporal locality to push subparts of the model to the edge. This approach is embodied in a distributed caching framework, Cachier. Cachier is built upon a novel caching model for recognition, and is distributed across edge servers and devices. The analytical caching model for recognition provides a formulation for expected latency for recognition requests in Cachier. The formulation incorporates the effects of compute time and accuracy. It also incorporates network conditions, thus providing a method to compute expected response times under various conditions. This is utilized as a cost function by Cachier, at edge servers and devices. By analyzing requests at the edge server, Cachier caches relevant parts of the trained model at edge servers, which is used to respond to requests, minimizing the number of requests that go to the cloud. Then, Cachier uses context-aware prediction to prefetch parts of the trained model onto devices. The requests can then be processed on the devices, thus minimizing the number of offloaded requests. Finally, Cachier enables cooperation between nearby devices to allow exchanging prefetched data, reducing the dependence on remote servers even further. The efficacy of Cachier is evaluated by using it with an art recognition application. The application is driven using real world traces gathered at museums. By conducting a large-scale study with different control variables, we show that Cachier can lower latency, increase scalability and decrease infrastructure resource usage, while maintaining high accuracy.

Many banking and commerce mobile applications use two-factor authentication for userauthentication purposes which include both password and behavioral based authenticationsystems. These behavioral based authentication systems use different behavioral parametersrelated to typing behavior of the user and the way user handles the phone while typing. Theydistinguish users and impostors using machine learning techniques (mostly supervised learningtechniques) on these behavioral data. Both password and behavior based systems work well indetecting imposters on mobile applications, but they can suffer from record and replay attackswhere the touch related information of the user actions is recorded and replayedprogrammatically. These are called as Record & Replay (R & R) bots. The effectiveness ofbehavioral authentication systems in identifying such attacks is unexplored. The current thesiswork tries to address this problem by developing a method to identify R & R bots on mobileapplications. In this work, behavioral data from users and corresponding R & R bot is collectedand it is observed that the touch information (location of touch on the screen, touch pressure,area of finger in contact with screen) is exactly replayed by the bot. However, sensorinformation seemed to be different in the case of user and corresponding R & R bot where thephysical touch action misses while replaying user actions on the mobile application. Based onthis observation, a feature set is extracted from the sensor data that can be used to differentiateusers from bots and a dataset is formed which contains the data corresponding to these featuresfrom both users and bots. Two machine learning techniques namely support vector machines(SVM) and logistic regression (LR) are applied on the training dataset (80% of the dataset) tobuild classifiers. The two classifiers built using the training dataset are able to classify user andbot sessions accurately in the test dataset (20% of the dataset) based on the feature set derivedfrom the sensor data.