No description available.

In the environment where new large scale problems are emerging in various disciplines and pervasive computing applications are becoming more common, there is an urgent need for machine learning algorithms that could process increasing amounts of data using comparatively smaller computing resources in a computational efficient way. Previous research has resulted in many successful learning algorithms that scale linearly or even sub-linearly with sample size and dimension, both in runtime and in space. However, linear or even sub-linear space scaling is often not sufficient, because it implies an unbounded growth in memory with sample size. This clearly opens another challenge: how to learn from large, or practically infinite, data sets or data streams using memory limited resources. Online learning is an important learning scenario in which a potentially unlimited sequence of training examples is presented one example at a time and can only be seen in a single pass. This is opposed to offline learning where the whole collection of training examples is at hand. The objective is to learn an accurate prediction model from the training stream. Upon on repetitively receiving fresh example from stream, typically, online learning algorithms attempt to update the existing model without retraining. The invention of the Support Vector Machines (SVM) attracted a lot of interest in adapting the kernel methods for both offline and online learning. Typical online learning for kernel classifiers consists of observing a stream of training examples and their inclusion as prototypes when specified conditions are met. However, such procedure could result in an unbounded growth in the number of prototypes. In addition to the danger of the exceeding the physical memory, this also implies an unlimited growth in both update and prediction time. To address this issue, in my dissertation I propose a series of kernel-based budgeted online algorithms, which have constant space and constant update and prediction time. This is achieved by maintaining a fixed number of prototypes under the memory budget. Most of the previous works on budgeted online algorithms focus on kernel perceptron. In the first part of the thesis, I review and discuss these existing algorithms and then propose a kernel perceptron algorithm which removes the prototype with the minimal impact on classification accuracy to maintain the budget. This is achieved by dual use of cached prototypes for both model presentation and validation. In the second part, I propose a family of budgeted online algorithms based on the Passive-Aggressive (PA) style. The budget maintenance is achieved by introducing an additional constraint into the original PA optimization problem. A closed-form solution was derived for the budget maintenance and model update. In the third part, I propose a budgeted online SVM algorithm. The proposed algorithm guarantees that the optimal SVM solution is maintained on all the prototype examples at any time. To maximize the accuracy, prototypes are constructed to approximate the data distribution near the decision boundary. In the fourth part, I propose a family of budgeted online algorithms for multi-class classification. The proposed algorithms are the recently proposed SVM training algorithm Pegasos. I prove that the gap between the budgeted Pegasos and the optimal SVM solution directly depends on the average model degradation due to budget maintenance. Following the analysis, I studied greedy multi-class budget maintenance methods based on removal, projection and merging of SVs. In each of these four parts, the proposed algorithms were experimentally evaluated against the state-of-art competitors. The results show that the proposed budgeted online algorithms outperform the competitive algorithm and achieve accuracy comparable to non-budget counterparts while being extremely computationally efficient. / Computer and Information Science

Yes / Face recognition is a very current subject of great interest in the area of visual computing. In the past, numerous face recognition and authentication approaches have been proposed, though the great majority of them use full frontal faces both for training machine learning algorithms and for measuring the recognition rates. In this paper, we discuss some novel experiments to test the performance of machine learning, especially the performance of deep learning, using partial faces as training and recognition cues. Thus, this study sharply differs from the common approaches of using the full face for recognition tasks. In particular, we study the rate of recognition subject to the various parts of the face such as the eyes, mouth, nose and the forehead. In this study, we use a convolutional neural network based architecture along with the pre-trained VGG-Face model to extract features for training. We then use two classifiers namely the cosine similarity and the linear support vector machine to test the recognition rates. We ran our experiments on the Brazilian FEI dataset consisting of 200 subjects. Our results show that the cheek of the face has the lowest recognition rate with 15% while the (top, bottom and right) half and the 3/4 of the face have near 100% recognition rates. / Supported in part by the European Union's Horizon 2020 Programme H2020-MSCA-RISE-2017, under the project PDE-GIR with grant number 778035.

Alcohol use disorder (AUD) remains a major public health concern globally with substantially increased mortality and a significant economic burden. The low rates of treatment and the high rates of relapse mean that excessive alcohol consumption detrimentally affects many aspects of the user's life and the lives of those around them. One reason for the low efficacy of treatments for AUD could be an unclear understanding of the neural correlates of the disease. As such, the studies in this dissertation aim at elucidating the neural mechanisms undergirding AUD, which could lead to more efficacious treatment and rehabilitation strategies. The propensity for impulsive decision making (choosing smaller, sooner rewards over larger, later ones) also known as delay discounting (DD), is an established risk-factor for a variety of substance abuse disorders, including AUD. Brain mapping of DD routinely uses modalities such as blood-oxygenation-level-dependent functional magnetic resonance imaging (BOLD fMRI). However, the extent to which these brain activation maps reflect the characteristics of impulsive behavior has not been directly studied. To examine this, we used multi-voxel pattern analysis (MVPA) methods such as multivariate classification using Support Vector Machine (SVM) algorithms and trained accurate classifiers of high vs. low impulsivity with individual fMRI brain maps. Our results demonstrate that brain regions in the prefrontal cortex encode neuroeconomic decision making characterizing DD behavior and help classify individuals with low impulsivity from individuals with high impulsivity. Individuals suffering from addictive afflictions such as AUD are often unable to plan for the future and are trapped in a narrow temporal window, resulting in short-term, impulsive decision making. Episodic future thinking (EFT) or the ability to project oneself into the future and pre-experience an event, is a rapidly growing area of addiction research and individuals suffering from addictive disorders are often poor at it. However, it has been shown across healthy individuals and disease populations (addiction, obesity) that practicing EFT reduces impulsive decision making. We provided real-time fMRI neurofeedback to alcohol users while they performed EFT inside the MR scanner to aid them in successfully modulating their thoughts between the present and the future. After the scanning session, participants made more restrained choices when performing a behavioral task outside the scanner, demonstrating an improvement in impulsivity. These two neuroimaging studies interrogate the brain mechanisms of delay discounting and episodic future thinking in alcohol use disorder. Successful classification of impulsive behavior as demonstrated in the first study could lead to accurate prediction of treatment outcomes in AUD. The second study suggests that rtfMRI provides direct access to brain mechanisms regulating EFT and highlights its potential as an intervention for impulsivity in the context of AUD. The work in this dissertation thus investigates important cognitive process for the treatment of alcohol use disorder that could pave the way for novel therapeutic interventions not only for AUD, but also for a wide spectrum of other addictive disorders. / Doctor of Philosophy / Alcohol use disorder (AUD) remains a major public health concern globally with substantially increased mortality and a significant economic burden. The low rates of treatment and the high rates of relapse mean that excessive alcohol consumption detrimentally affects many aspects of the user’s life and the lives of those around them. One reason for the low efficacy of treatments for AUD could be an unclear understanding of the brain regions affected by it. As such, the studies in this dissertation aim at elucidating the neural mechanisms undergirding AUD, which could lead to more efficacious treatment and rehabilitation strategies. The propensity for impulsive decision making (choosing smaller, sooner rewards over larger, later ones) also known as delay discounting (DD), is an established risk-factor for a variety of substance abuse disorders, including AUD. Brain mapping of DD routinely uses modalities such as blood-oxygenation-level-dependent functional magnetic resonance imaging (BOLD fMRI). However, the extent to which these brain activation maps reflect the characteristics of impulsive behavior has not been directly studied. To examine this, we searched for highly reproducible spatial patterns of brain activation that differ across experimental conditions (multi-voxel pattern analysis) and trained accurate classifiers of high vs. low impulsivity with individual fMRI brain maps. Our results demonstrate that brain regions in the prefrontal cortex encode neuroeconomic decision making and help classify individuals with low impulsivity from individuals with high impulsivity. Individuals suffering from addictive afflictions such as AUD are often unable to plan for the future and are trapped in a narrow temporal window, resulting in short-term, impulsive decision making. Episodic future thinking (EFT) or the ability to project oneself into the future and pre-experience an event, is a rapidly growing area of addiction research. However, it has been shown across healthy individuals and disease populations (addiction, obesity) that practicing EFT reduces impulsive decision making. We provided v real-time fMRI neurofeedback to alcohol users while they performed EFT inside the MR scanner to aid them in successfully modulating their thoughts between the present and the future. After the scanning session, participants made more restrained choices when performing a behavioral task outside the scanner, demonstrating an improvement in impulsivity. These two neuroimaging studies interrogate the brain mechanisms of delay discounting and episodic future thinking in alcohol use disorder. Successful classification of impulsive behavior as demonstrated in the first study could lead to accurate prediction of treatment outcomes in AUD. The second study suggests that rtfMRI provides direct access to brain mechanisms regulating EFT and highlights its potential as an intervention for impulsivity in the context of AUD. The work in this dissertation thus investigates important cognitive process for the treatment of alcohol use disorder that could pave the way for novel therapeutic interventions not only for AUD, but also for a wide spectrum of other addictive disorders.

We have explored correlations between the measured efficiency of the RNAi process and several computed signatures that characterize equilibrium secondary structure of the participating mRNA, siRNA, and their complexes. A previously published data set of 609 experimental points was used for the analysis. While virtually no correlation with the computed structural signatures are observed for individual data points, several clear trends emerge when the data is averaged over 10 bins of N ~ 60 data points per bin. The strongest trend is a positive linear (r² = 0.87) correlation between ln(remaining mRNA) and ΔG_ms, the combined free energy cost of unraveling the siRNA and creating the break in the mRNA secondary structure at the complementary target strand region. At the same time, the free energy change ΔG_total of the entire process mRNA + siRNA → (mRNA – siRNA)_complex is not correlated with RNAi efficiency, even after averaging. These general findings appear to be robust to details of the computational protocols. The correlation between computed ΔG_ms and experimentally observed RNAi efficiency can be used to enhance the ability of a machine learning algorithm based on a support vector machine (SVM) to predict effective siRNA sequences for a given target mRNA. Specifically, we observe modest, 3 to 7%, but consistent improvement in the positive predictive value (PPV) when the SVM training set is pre- or post-filtered according to a ΔG_ms threshold. / Master of Science

The current practice of construction productivity and emission monitoring is performed by either manual stopwatch studies which are significantly labor intensive and subject to human errors, or by the use of RFID and GPS tracking devices which may be costly and impractical. To address these limitations, a novel computer vision based method for automated 2D tracking, 3D localization, and action recognition of construction equipment from different camera viewpoints is presented. In the proposed method, a new algorithm based on Histograms of Oriented Gradients and hue-saturation Colors (HOG+C) is used for 2D tracking of the earthmoving equipment. Once the equipment is detected, using a Direct Linear Transformation followed by a non-linear optimization, their positions are localized in 3D. In order to automatically analyze the performance of these operations, a new algorithm to recognize actions of the equipment is developed. First, a video is represented as a collection of spatio-temporal features by extracting space-time interest points and describing each with a Histogram of Oriented Gradients (HOG). The algorithm automatically learns the distributions of these features by clustering their HOG descriptors. Equipment action categories are then learned using a multi-class binary Support Vector Machine (SVM) classifier. Given a novel video sequence, the proposed method recognizes and localizes equipment actions. The proposed method has been exhaustively tested on 859 videos from earthmoving operations. Experimental results with an average accuracy of 86.33% and 98.33% for excavator and truck action recognition respectively, reflect the promise of the proposed method for automated performance monitoring. / Master of Science

Fixed-object crashes have long time been considered as major roadway safety concerns. While previous relevant studies tended to address such crashes in the context of roadway departures, and heavily relied on police-reported accidents data, this study integrated the SHRP2 NDS and RID data for analyses, which fully depicted the prior to, during, and after crash scenarios. A total of 1,639 crash, near-crash events, and 1,050 baseline events were acquired. Three analysis methods: logistic regression, support vector machine (SVM) and artificial neural network (ANN) were employed for two responses: crash occurrence and severity level. Logistic regression analyses identified 16 and 10 significant variables with significance levels of 0.1, relevant to driver, roadway, environment, etc. for two responses respectively. The logistic regression analyses led to a series of findings regarding the effects of explanatory variables on fixed-object event occurrence and associated severity level. SVM classifiers and ANN models were also constructed to predict these two responses. Sensitivity analyses were performed for SVM classifiers to infer the contributing effects of input variables. All three methods obtained satisfactory prediction performance, that was around 88% for fixed-object event occurrence and 75% for event severity level, which indicated the effectiveness of NDS event data on depicting crash scenarios and roadway safety analyses. / Master of Science / Fixed-object crashes happen when a single vehicle strikes a roadway feature such as a curb or a median, or runs off the road and hits a roadside feature such as a tree or utility pole. They have long time been considered as major highway safety concerns due to their high frequency, fatality rate, and associated property cost. Previous studies relevant to fixed-object crashes tended to address such crashes in the contexture of roadway departures, and heavily relied on police-reported accident data. However, many fixed-object crashes involved objects in roadway such as traffic control devices, roadway debris, etc. The police-reported accident data were found to be weak in depicting scenarios prior to, during crashes. Also, many minor crashes were often kept unreported. The Second Strategic Highway Research Program (SHRP2) Naturalistic Driving Study (NDS) is the largest NDS project launched across the country till now, aimed to study driver behavior or, performance-related safety problems under real-world scenarios. The data acquisition systems (DASs) equipped on participated vehicles collect vehicle kinematics, roadway, traffic, environment, and driver behavior data continuously, which enable researchers to address such crash scenarios closely. This study integrated SHRP2 NDS and roadway information database (RID) data to conduct a comprehensive analysis of fixed-object crashes. A total of 1,639 crash, near-crash events relevant to fixed objects and animals, and 1,050 baseline events were used. Three analysis methods: logistic regression, support vector machine (SVM) and artificial neural network (ANN) were employed for two responses: crash occurrence and severity level. The logistic regression analyses identified 16 and 10 variables with significance levels of 0.1 for fixed-object event occurrence and severity level models respectively. The influence of explanatory variables was discussed in detail. SVM classifiers and ANN models were also constructed to predict the fixed-object crash occurrence and severity level. Sensitivity analyses were performed for SVM classifiers to infer the contributing effects of input variables. All three methods achieved satisfactory prediction accuracies of around 88% for crash occurrence prediction and 75% for crash severity level prediction, which suggested the effectiveness of NDS event data on depicting crash scenarios and roadway safety analyses.

Metabolomics aims at the detection and quantitation of metabolites within a biological system. As the most direct representation of phenotypic changes, metabolomics is an important component in system biology research. Recent development on high-resolution, high-accuracy mass spectrometers enables the simultaneous study of hundreds or even thousands of metabolites in one experiment. Liquid chromatography-mass spectrometry (LC-MS) is a commonly used instrument for metabolomic studies due to its high sensitivity and broad coverage of metabolome. However, the identification of metabolites remains a bottle-neck for current metabolomic studies. This thesis focuses on utilizing computational approaches to improve the accuracy and efficiency for metabolite identification in LC-MS/MS-based metabolomic studies. First, an outlier screening approach is developed to identify those LC-MS runs with low analytical quality, so they will not adversely affect the identification of metabolites. The approach is computationally simple but effective, and does not depend on any preprocessing approach. Second, an integrated computational framework is proposed and implemented to improve the accuracy of metabolite identification and prioritize the multiple putative identifications of one peak in LC-MS data. Through the framework, peaks are likely to have the m/z values that can give appropriate putative identifications. And important guidance for the metabolite verification is provided by prioritizing the putative identifications. Third, an MS/MS spectral matching algorithm is proposed based on support vector machine classification. The approach provides an improved retrieval performance in spectral matching, especially in the presence of data heterogeneity due to different instruments or experimental settings used during the MS/MS spectra acquisition. / Master of Science

La sélection de modèle est un thème majeur de l'apprentissage statistique. Dans ce manuscrit, nous introduisons des méthodes de sélection de modèle dédiées à des SVM bi-classes et multi-classes. Ces machines ont pour point commun d'être à coût quadratique, c'est-à-dire que le terme empirique de la fonction objectif de leur problème d'apprentissage est une forme quadratique. Pour les SVM, la sélection de modèle consiste à déterminer la valeur optimale du coefficient de régularisation et à choisir un noyau approprié (ou les valeurs de ses paramètres). Les méthodes que nous proposons combinent des techniques de parcours du chemin de régularisation avec de nouveaux critères de sélection. La thèse s'articule autour de trois contributions principales. La première est une méthode de sélection de modèle par parcours du chemin de régularisation dédiée à la l2-SVM. Nous introduisons à cette occasion de nouvelles approximations de l'erreur en généralisation. Notre deuxième contribution principale est une extension de la première au cas multi-classe, plus précisément à la M-SVM². Cette étude nous a conduits à introduire une nouvelle M-SVM, la M-SVM des moindres carrés. Nous présentons également de nouveaux critères de sélection de modèle pour la M-SVM de Lee, Lin et Wahba à marge dure (et donc la M-SVM²) : un majorant de l'erreur de validation croisée leave-one-out et des approximations de cette erreur. La troisième contribution principale porte sur l'optimisation des valeurs des paramètres du noyau. Notre méthode se fonde sur le principe de maximisation de l'alignement noyau/cible, dans sa version centrée. Elle l'étend à travers l'introduction d'un terme de régularisation. Les évaluations expérimentales de l'ensemble des méthodes développées s'appuient sur des benchmarks fréquemment utilisés dans la littérature, des jeux de données jouet et des jeux de données associés à des problèmes du monde réel / Model selection is of major interest in statistical learning. In this document, we introduce model selection methods for bi-class and multi-class support vector machines. We focus on quadratic loss machines, i.e., machines for which the empirical term of the objective function of the learning problem is a quadratic form. For SVMs, model selection consists in finding the optimal value of the regularization coefficient and choosing an appropriate kernel (or the values of its parameters). The proposed methods use path-following techniques in combination with new model selection criteria. This document is structured around three main contributions. The first one is a method performing model selection through the use of the regularization path for the l2-SVM. In this framework, we introduce new approximations of the generalization error. The second main contribution is the extension of the first one to the multi-category setting, more precisely the M-SVM². This study led us to derive a new M-SVM, the least squares M-SVM. Additionally, we present new model selection criteria for the M-SVM introduced by Lee, Lin and Wahba (and thus the M-SVM²). The third main contribution deals with the optimization of the values of the kernel parameters. Our method makes use of the principle of kernel-target alignment with centered kernels. It extends it through the introduction of a regularization term. Experimental validation of these methods was performed on classical benchmark data, toy data and real-world data

Esta tese propõe uma metodologia de detecção de anomalias por meio da otimização da extração, seleção e classificação de características relacionadas ao tráfego de redes Real Time Ethernet (RTE). Em resumo, dois classificadores são treinados usando características que são extraídas do tráfego por meio da técnica de janela deslizante e posteriormente selecionadas de acordo com sua correlação com o evento a ser classificado. O número de características relevantes pode variar de acordo com os indicadores de desempenho de cada classificador. Reduzindo a dimensionalidade do evento a ser classificado com o menor número de características possíveis que o represente, são garantidos a redução do esforço computacional, ganho de tempo, dentre outros benefícios. Posteriormente, os classificadores são comparados em função dos indicadores de desempenho: acurácia, taxa de falsos positivos, taxa de falsos negativos, tempo de processamento e erro relativo. A metodologia proposta foi utilizada para identificar quatro diferentes eventos (três anomalias e o estado normal de operação) em redes PROFINET reais e com configurações distintas entre si; também foi aplicada em três eventos (duas anomalias e o estado normal de operação) em redes SERCOS III. O desempenho de cada classificador é analisado em suas particularidades e comparados com pesquisas correlatas. Por fim, é explorada a possibilidade de aplicação da metodologia proposta para outros protocolos baseados em RTE. / This thesis proposes an anomaly detection methodology by optimizing extraction, selection and classification of characteristics related to Real Time Ethernet (RTE) network traffic. In summary, two classifiers are trained using features which are extracted from network traffic through the sliding window technique and selected according to their correlation with the event being classified. The number of relevant characteristics could vary according to performance indicators of each classifier. Reducing the dimensionality of the event to be classified using the smallest number of characteristics which represent it, guarantees reduction in computational effort, processing time, among other benefits. The classifiers are compared according to performance indicators: accuracy, false positive rate, false negative rate, processing time and relative error. The proposed methodology was used to identify four different events (three anomalies and normal operation) in real PROFINET networks, using different configurations. It was also applied in 3 events (two anomalies and normal operation) in SERCOS III networks. The results obtained are analyzed in its particularities and compared with related research. Finally, the possibility of applying the proposed methodology for other protocols based on RTE is explored.