This dissertation describes a study which aims to classify video flows from Internet network traffic. In this study, classification is done based on the characteristics of the flow, which includes features such as payload sizes and inter-arrival time. The purpose of this is to give an alternative to classifying flows based on the contents of their payload packets. Because of an increase of encrypted flows within Internet network traffic, this is a necessity. Data with known class is fed to a machine learning classifier such that a model can be created. This model can then be used for classification of new unknown data. For this study, two different classifiers are used, namely decision trees and random forest. Several tests are completed to attain the best possible models. The results of this dissertation shows that classification based on characteristics is possible and the random forest classifier in particular achieves good accuracies. However, the accuracy of classification of encrypted flows was not able to be tested within this project. / HITS, 4707

Thesis advisor: Ziqiang . Wang / The transition between many-body localized states and the delocalized thermal states is an eigenstate phase transition at finite energy density outside the scope of conventional quantum statistical mechanics. We apply support vector machine (SVM) to study the phase transition between many-body localized and thermal phases in a disordered quantum Ising chain in a transverse external field. The many-body eigenstate energy E is bounded by a bandwidth W=Eₘₐₓ-Eₘᵢₙ. The transition takes place on a phase diagram spanned by the energy density ϵ=2(Eₘₐₓ-Eₘᵢₙ)/W and the disorder strength ẟJ of the spin interaction uniformly distributed within [-ẟJ, ẟJ], formally parallel to the mobility edge in Anderson localization. In our study we use the labeled probability density of eigenstate wavefunctions belonging to the deeply localized and thermal regimes at two different energy densities (ϵ's) as the training set, i.e., providing labeled data at four corners of the phase diagram. Then we employ the trained SVM to predict the whole phase diagram. The obtained phase boundary qualitatively agrees with previous work using entanglement entropy to characterize these two phases. We further analyze the decision function of the SVM to interpret its physical meaning and find that it is analogous to the inverse participation ratio in configuration space. Our findings demonstrate the ability of the SVM to capture potential quantities that may characterize the many-body localization phase transition. To further investigate the properties of the transition, we study the behavior of the entanglement entropy of a subsystem of size L_A in a system of size L > L_A near the critical regime of the many-body localization transition. The many-body eigenstates are obtained by exact diagonalization of a disordered quantum spin chain under twisted boundary conditions to reduce the finite-size effect. We present a scaling theory based on the assumption that the transition is continuous and use the subsystem size L_A/ξ as the scaling variable, where ξ is the correlation length. We show that this scaling theory provides an effective description of the critical behavior and that the entanglement entropy follows the thermal volume law at the transition point. We extract the critical exponent governing the divergence of ξ upon approaching the transition point. We again study the participation entropy in the spin-basis of the domain wall excitations and show that the transition point and the critical exponent agree with those obtained from finite size scaling of the entanglement entropy. Our findings suggest that the many-body localization transition in this model is continuous and describable as a localization transition in the many-body configuration space. Besides the many-body localization transition driven by disorder, We also study the Coulomb repulsion and temperature driving phase transitions. We apply a finite-temperature Gutzwiller projection to two-dimensional Hubbard model by constructing a "Gutzwiller-type" density matrix operator to approximate the real interacting density matrix, which provides the upper bound of free energy of the system. We firstly investigate half filled Hubbard model without magnetism and obtain the phase diagram. The transition line is of first order at finite temperature, ending at 2 second order points, which shares qualitative agreement with dynamic mean field results. We derive the analytic form of the free energy and therefor the equation of states, which benefits the understanding of the different phases. We later extend our approach to take anti-ferromagnetic order into account. We determine the Neel temperature and explore its interesting behavior when varying the Coulomb repulsion. / Thesis (PhD) — Boston College, 2019. / Submitted to: Boston College. Graduate School of Arts and Sciences. / Discipline: Physics.

As machine learning becomes an increasingly relevant field being incorporated into everyday life, so does the need for consistently high performing models. With these high expectations, along with potentially restrictive data sets, it is crucial to be able to use techniques for machine learning that increase the likelihood of success. Robust Principal Component Analysis (RPCA) not only extracts anomalous data, but also finds correlations among the given features in a data set, in which these correlations can themselves be used as features. By taking a novel approach to utilizing the output from RPCA, we address how our method effects the performance of such models. We take into account the efficiency of our approach, and use projectors to enable our method to have a 99.79% faster run time. We apply our method primarily to cyber security data sets, though we also investigate the effects on data sets from other fields (e.g. medical).

As online casino and betting increases in popularity across the globe, the importance of green gambling has become an important subject of discussion. The Swedish betting company, ATG, realises the benefits of this and would like to prevent their gamblers from falling into problem gambling. To predict problem gambling, Bayesian networks were trained on previously identified problem gamblers, separated into seven risk groups. The network was then able to predict the risk group of previously unseen gamblers with an ac- curacy of 94%. It also achieved an average precision of 89%, an average recall of 96% and an average f1-score of 93%. The features in the data set were also ranked, to find which were most important in predicting problem gambling. It was found that municipality, which day of the week the transaction was made and during which hour of the day were the most important features. Also, the Bayesian network was also made as simple as possible, by removing irrelevant features and features which carry very low importance.

My dissertation, titled “From the machine mind to the human mind: using machine learning to understand (ir)rationality, bias and polarization in human beings,” investigates ways in which human minds operate and seeks to uncover the causes of biasedness, limited rationality, and polarization of human minds, to eventually devise tools to compensate for such human limitations. Chapter 2 of the thesis focuses on the evaluation of information and decision making under enormous information asymmetry, in the setting of patients evaluating doctors’ medical advice. Patients were found to be poor evaluators who were unable to distinguish good from bad due to their lack of medical expertise, and unable to overcome their own irrationality and bias. I emphasize the ramification of such limited rationality, which might lead to the adoption of suboptimal or bad medical opinions, and propose ways to improve this situation by redesigning some features of the platform, and/or implementing new policies to help good doctors on the platform. Chapter 3 focuses on developing a new metric that reliably measures the ideology of the US elites. This metric was developed based on congressional reports which made it unique and relatively independent from established metrics based on roll call votes, such as DW-NOMINATE. First, I leveraged a neural network-based approach to decompose the speech documents into frames and topics components, with all ideological information funneled into the frames component. Eventually, two different ideology metrics were obtained and validated: an embedding vector and an ideological slant score. Later I showed that our new metrics can predict party switchers and trespassers with high recall. In chapter 4, I applied the newly obtained metric (mainly slant scores) to investigate various aspects of the congress, such as the heterogeneity of ideology among the members, the temporal evolution of partisan division, the bill passing, and the re-election strategy of the senators.

abstract: Students seldom spontaneously collaborate with each other. A system that can measure collaboration in real time could be useful, for example, by helping the teacher locate a group requiring guidance. To address this challenge, the research presented here focuses on building and comparing collaboration detectors for different types of classroom problem solving activities, such as card sorting and handwriting. Transfer learning using different representations was also studied with a goal of building collaboration detectors for one task can be used with a new task. Data for building such detectors were collected in the form of verbal interaction and user action logs from students’ tablets. Three qualitative levels of interactivity were distinguished: Collaboration, Cooperation and Asymmetric Contribution. Machine learning was used to induce a classifier that can assign a code for every episode based on the set of features. The results indicate that machine learned classifiers were reliable and can transfer. / Dissertation/Thesis / Doctoral Dissertation Computer Science 2020

Most real-world data such as images and videos are naturally organized as tensors, and often have high dimensionality. Tensor subspace learning is a fundamental problem that aims at finding low-dimensional representations from tensors while preserving their intrinsic characteristics. By dealing with tensors in the learned subspace, subsequent tasks such as clustering, classification, visualization, and interpretation can be greatly facilitated. This thesis studies the tensor subspace learning problem from a generative perspective, and proposes four probabilistic methods that generalize the ideas of classical subspace learning techniques for tensor analysis. Probabilistic Rank-One Tensor Analysis (PROTA) generalizes probabilistic principle component analysis. It is flexible in capturing data characteristics, and avoids rotational ambiguity. For robustness against overfitting, concurrent regularizations are further proposed to concurrently and coherently penalize the whole subspace, so that unnecessary scale restrictions can be relaxed in regularizing PROTA. Probabilistic Rank-One Discriminant Analysis (PRODA) is a bilinear generalization of probabilistic linear discriminant analysis. It learns a discriminative subspace by representing each observation as a linear combination of collective and individual rank-one matrices. This provides PRODA with both the expressiveness of capturing discriminative features and non-discriminative noise, and the capability of exploiting the (2D) tensor structures. Bilinear Probabilistic Canonical Correlation Analysis (BPCCA) generalizes probabilistic canonical correlation analysis for learning correlations between two sets of matrices. It is built on a hybrid Tucker model in which the two-view matrices are combined in two stages via matrix-based and vector-based concatenations, respectively. This enables BPCCA to capture two-view correlations without breaking the matrix structures. Bayesian Low-Tubal-Rank Tensor Factorization (BTRTF) is a fully Bayesian treatment of robust principle component analysis for recovering tensors corrupted with gross outliers. It is based on the recently proposed tensor-SVD model, and has more expressive modeling power in characterizing tensors with certain orientation such as images and videos. A novel sparsity-inducing prior is also proposed to provide BTRTF with automatic determination of the tensor rank (subspace dimensionality). Comprehensive validations and evaluations are carried out on both synthetic and real-world datasets. Empirical studies on parameter sensitivities and convergence properties are also provided. Experimental results show that the proposed methods achieve the best overall performance in various applications such as face recognition, photograph-sketch match, and background modeling. Keywords: Tensor subspace learning, probabilistic models, Bayesian inference, tensor decomposition.

Graph Convolutional Networks (GCNs) draws significant attention due to its power of representation learning on graphs. Recent works developed frameworks to train deep GCNs. Such works show impressive results in tasks like point cloud classification and segmentation, and protein interaction prediction. While for large-scale graphs, doing full-batch training by GCNs is still challenging especially when GCNs go deeper. By fully analyzing a clustering-based mini-batch training algorithm ClusterGCN, we propose random partition which is a more efficient and effective method to implement mini-batch training. Besides, selecting different permutation invariance function (such as max, mean or add) for neighbors’ information aggregation will result in every different results. Therefore, we propose to alleviate it by introducing a novel Generalized Aggregation Function. In this thesis, I analyze the drawbacks caused by ClusterGCN and discuss about its limits. I further compare the performance of ClusterGCN with random partition and the final experimental results show that simple random partition outperforms ClusterGCN with very obvious advantageous for node property prediction task. For the techniques which are commonly used to make GCNs go deeper, I demonstrate a better way of applying residual connections (pre-activation) to stack more layers for GCNs. Last, I show the complete work of training deeper GCNs with generalized aggregators and display the promising results over several datasets from the Open Graph Benchmark (OGB).

No description available.

abstract: In recent years, the widespread use of deep neural networks (DNNs) has facilitated great improvements in performance for computer vision tasks like image classification and object recognition. In most realistic computer vision applications, an input image undergoes some form of image distortion such as blur and additive noise during image acquisition or transmission. Deep networks trained on pristine images perform poorly when tested on such distortions. DNN predictions have also been shown to be vulnerable to carefully crafted adversarial perturbations. Specifically, so-called universal adversarial perturbations are image-agnostic perturbations that can be added to any image and can fool a target network into making erroneous predictions. This work proposes selective DNN feature regeneration to improve the robustness of existing DNNs to image distortions and universal adversarial perturbations. In the context of common naturally occurring image distortions, a metric is proposed to identify the most susceptible DNN convolutional filters and rank them in order of the highest gain in classification accuracy upon correction. The proposed approach called DeepCorrect applies small stacks of convolutional layers with residual connections at the output of these ranked filters and trains them to correct the most distortion-affected filter activations, whilst leaving the rest of the pre-trained filter outputs in the network unchanged. Performance results show that applying DeepCorrect models for common vision tasks significantly improves the robustness of DNNs against distorted images and outperforms other alternative approaches. In the context of universal adversarial perturbations, departing from existing defense strategies that work mostly in the image domain, a novel and effective defense which only operates in the DNN feature domain is presented. This approach identifies pre-trained convolutional features that are most vulnerable to adversarial perturbations and deploys trainable feature regeneration units which transform these DNN filter activations into resilient features that are robust to universal perturbations. Regenerating only the top 50% adversarially susceptible activations in at most 6 DNN layers and leaving all remaining DNN activations unchanged can outperform existing defense strategies across different network architectures and across various universal attacks. / Dissertation/Thesis / Doctoral Dissertation Electrical Engineering 2020