<p>Subspaces and manifolds are two powerful models for high dimensional signals. Subspaces model linear correlation and are a good fit to signals generated by physical systems, such as frontal images of human faces and multiple sources impinging at an antenna array. Manifolds model sources that are not linearly correlated, but where signals are determined by a small number of parameters. Examples are images of human faces under different poses or expressions, and handwritten digits with varying styles. However, there will always be some degree of model mismatch between the subspace or manifold model and the true statistics of the source. This dissertation exploits subspace and manifold models as prior information in various signal processing and machine learning tasks.</p><p>A near-low-rank Gaussian mixture model measures proximity to a union of linear or affine subspaces. This simple model can effectively capture the signal distribution when each class is near a subspace. This dissertation studies how the pairwise geometry between these subspaces affects classification performance. When model mismatch is vanishingly small, the probability of misclassification is determined by the product of the sines of the principal angles between subspaces. When the model mismatch is more significant, the probability of misclassification is determined by the sum of the squares of the sines of the principal angles. Reliability of classification is derived in terms of the distribution of signal energy across principal vectors. Larger principal angles lead to smaller classification error, motivating a linear transform that optimizes principal angles. This linear transformation, termed TRAIT, also preserves some specific features in each class, being complementary to a recently developed Low Rank Transform (LRT). Moreover, when the model mismatch is more significant, TRAIT shows superior performance compared to LRT.</p><p>The manifold model enforces a constraint on the freedom of data variation. Learning features that are robust to data variation is very important, especially when the size of the training set is small. A learning machine with large numbers of parameters, e.g., deep neural network, can well describe a very complicated data distribution. However, it is also more likely to be sensitive to small perturbations of the data, and to suffer from suffer from degraded performance when generalizing to unseen (test) data.</p><p>From the perspective of complexity of function classes, such a learning machine has a huge capacity (complexity), which tends to overfit. The manifold model provides us with a way of regularizing the learning machine, so as to reduce the generalization error, therefore mitigate overfiting. Two different overfiting-preventing approaches are proposed, one from the perspective of data variation, the other from capacity/complexity control. In the first approach, the learning machine is encouraged to make decisions that vary smoothly for data points in local neighborhoods on the manifold. In the second approach, a graph adjacency matrix is derived for the manifold, and the learned features are encouraged to be aligned with the principal components of this adjacency matrix. Experimental results on benchmark datasets are demonstrated, showing an obvious advantage of the proposed approaches when the training set is small.</p><p>Stochastic optimization makes it possible to track a slowly varying subspace underlying streaming data. By approximating local neighborhoods using affine subspaces, a slowly varying manifold can be efficiently tracked as well, even with corrupted and noisy data. The more the local neighborhoods, the better the approximation, but the higher the computational complexity. A multiscale approximation scheme is proposed, where the local approximating subspaces are organized in a tree structure. Splitting and merging of the tree nodes then allows efficient control of the number of neighbourhoods. Deviation (of each datum) from the learned model is estimated, yielding a series of statistics for anomaly detection. This framework extends the classical {\em changepoint detection} technique, which only works for one dimensional signals. Simulations and experiments highlight the robustness and efficacy of the proposed approach in detecting an abrupt change in an otherwise slowly varying low-dimensional manifold.</p> / Dissertation

This paper explores the potential applications of existing spectral clustering algorithms to real life problems through experiments on existing road traffic data. The analysis begins with an overview of previous unsupervised machine learning techniques and constructs an effective spectral clustering algorithm that demonstrates the analytical power of the method. The paper focuses on the spectral embedding method’s ability to project non-linearly separable, high dimensional data into a more manageable space that allows for accurate clustering. The key step in this method involves solving a normalized eigenvector problem in order to construct an optimal representation of the original data. While this step greatly enhances our ability to analyze the relationships between data points and identify the natural clusters within the original dataset, it is difficult to comprehend the eigenvalue representation of the data in terms of the original input variables. The later sections of this paper will explore how the careful framing of questions with respect to available data can help researchers extract tangible decision driving results from real world data through spectral clustering analysis.

Cloud computing, mobile computing, Service-Oriented Computing (SOC), and Software as a Service (SaaS) indicate that the Internet emerges to an anonymous service market where service functionality can be dynamically and ubiquitously consumed. Among functionally similar services, service consumers are interested in the consumption of the services which perform best towards their optimization preferences. The experienced performance of a service at consumer side is expressed in its non-functional properties (NFPs). Selecting the best-fit service is an individual challenge as the preferences of consumers vary. Furthermore, service markets such as the Internet are characterized by perpetual change and complexity. The complex collaboration of system environments and networks as well as expected and unexpected incidents may result in various performance experiences of a specific service at consumer side. The consideration of certain call side aspects that may distinguish such differences in the experience of NFPs is reflected in various call contexts. Service optimization based on a collaborative knowledge base of previous experiences of other, similar consumers with similar preferences is a desirable foundation. The research work described in this dissertation aims at an individually optimized selection of services considering the individual call contexts that have an impact on the performance, or NFPs in general, of a service as well as the various consumer preferences. The presented approach exploits shared measurement information about the NFP behavior of a service gained from former service calls of previous consumptions. Gaining selection/recommendation knowledge from shared experience benefits existing as well as new consumers of a service before its (initial) consumption. Our approach solely focuses on the optimization and collaborative information exchange among service consumers. It does not require the contribution of service providers or other non-consuming entities. As a result, the contribution among the participating entities also contributes to their own overall optimization benefit. With the initial focus on a single-tier optimization, we additionally provide a conceptual solution to a multi-tier optimization approach for which our recommendation framework is prepared in general. For a consumer-sided optimization, we conducted a literature study of conference papers of the last decade in order to find out what NFPs are relevant for the selection and consumption of services. The ranked results of this study represent what a broad scientific community determined to be relevant NFPs for service selection. We analyzed two general approaches for the employment of machine learning methods within our recommendation framework as part of the preparation of the actual recommendation knowledge. Addressing a future service market that has not fully developed yet and due to the fact that it seems to be impossible to be aware of the actual NFP data of different Web services at identical call contexts, a real-world validation is a challenge. In order to conduct an evaluation and also validation that can be considered to be close approximations to reality with the flexibility to challenge the machine learning approaches and methods as well as the overall recommendation approach, we used generated NFP data whose characteristics are influenced by measurement data gained from real-world Web services. For the general approach with the better evaluation results and benefits ratio, we furthermore analyzed, implemented, and validated machine learning methods that can be employed for service recommendation. Within the validation, we could achieve up to 95% of the overall achievable performance (utility) gain with a machine learning method that is focused on drift detection, which in turn, tackles the change characteristic of the Internet being an anonymous service market.

The use of predictive analysis is becoming more commonplace with each passing day, which lends increased credence to the fact that even governmental institutions should adopt it. Kronofogden is in the middle of a digitization process and is therefore in a unique position to implement predictive analysis into the core of their operations. This project aims to study if methods from predictive analysis can predict how many debts will be received for a first-time debtor, through the use of uplift modeling. The difference between uplift modeling and conventional modeling is that it aims to measure the difference in behavior after a treatment, in this case guidance from Kronofogden. Another aim of the project is to examine whether the scarce literature about uplift modeling have it right about how the conventional two-model approach fails to perform well in practical situations. The project shows similar results as Kronofogden’s internal evaluations. Three models were compared: random forests, gradient-boosted models and neural networks, the last performing the best. Positive uplift could be found for 1-5% of the debtors, meaning the current cutoff level of 15% is too high. The models have several potential sources of error, however: modeling choices, that the data might not be informative enough or that the actual expected uplift for new data is equal to zero.

Die Motivation für diese Dissertation ist dualer Natur: Einerseits ist die Dissertation methodologisch orientiert und entwickelt neue statistische Ansätze und Algorithmen für maschinelles Lernen. Gleichzeitig ist sie praktisch orientiert und fokussiert sich auf den konkreten Anwendungsfall von Produktretouren im Onlinehandel. Die “data explosion”, veursacht durch die Tatsache, dass die Kosten für das Speichern und Prozessieren großer Datenmengen signifikant gesunken sind (Bhimani and Willcocks, 2014), und die neuen Technologien, die daraus resultieren, stellen die größte Diskontinuität für die betriebliche Praxis und betriebswirtschaftliche Forschung seit Entwicklung des Internets dar (Agarwal and Dhar, 2014). Insbesondere die Business Intelligence (BI) wurde als wichtiges Forschungsthema für Praktiker und Akademiker im Bereich der Wirtschaftsinformatik (WI) identifiziert (Chen et al., 2012). Maschinelles Lernen wurde erfolgreich auf eine Reihe von BI-Problemen angewandt, wie zum Beispiel Absatzprognose (Choi et al., 2014; Sun et al., 2008), Prognose von Windstromerzeugung (Wan et al., 2014), Prognose des Krankheitsverlaufs von Patienten eines Krankenhauses (Liu et al., 2015), Identifikation von Betrug Abbasi et al., 2012) oder Recommender-Systeme (Sahoo et al., 2012). Allerdings gibt es nur wenig Forschung, die sich mit Fragestellungen um maschinelles Lernen mit spezifischen Bezug zu BI befasst: Obwohl existierende Algorithmen teilweise modifiziert werden, um sie auf ein bestimmtes Problem anzupassen (Abbasi et al., 2010; Sahoo et al., 2012), beschränkt sich die WI-Forschung im Allgemeinen darauf, existierende Algorithmen, die für andere Fragestellungen als BI entwickelt wurden, auf BI-Fragestellungen anzuwenden (Abbasi et al., 2010; Sahoo et al., 2012). Das erste wichtige Ziel dieser Dissertation besteht darin, einen Beitrag dazu zu leisten, diese Lücke zu schließen. Diese Dissertation fokussiert sich auf das wichtige BI-Problem von Produktretouren im Onlinehandel für eine Illustration und praktische Anwendung der vorgeschlagenen Konzepte. Viele Onlinehändler sind nicht profitabel (Rigby, 2014) und Produktretouren sind eine wichtige Ursache für dieses Problem (Grewal et al., 2004). Neben Kostenaspekten sind Produktretouren aus ökologischer Sicht problematisch. In der Logistikforschung ist es weitestgehend Konsens, dass die “letzte Meile” der Zulieferkette, nämlich dann wenn das Produkt an die Haustür des Kunden geliefert wird, am CO2-intensivsten ist (Browne et al., 2008; Halldórsson et al., 2010; Song et al., 2009). Werden Produkte retourniert, wird dieser energieintensive Schritt wiederholt, wodurch sich die Nachhaltigkeit und Umweltfreundlichkeit des Geschäftsmodells von Onlinehändlern relativ zum klassischen Vertrieb reduziert. Allerdings können Onlinehändler Produktretouren nicht einfach verbieten, da sie einen wichtigen Teil ihres Geschäftsmodells darstellen: So hat die Möglichkeit, Produkte zu retournieren positive Auswirkungen auf Kundenzufriedenheit (Cassill, 1998), Kaufverhalten (Wood, 2001), künftiges Kaufverhalten (Petersen and Kumar, 2009) und emotianale Reaktionen der Kunden (Suwelack et al., 2011). Ein vielversprechender Ansatz besteht darin, sich auf impulsives und kompulsives (LaRose, 2001) sowie betrügerisches Kaufverhalten zu fokussieren (Speights and Hilinski, 2005; Wachter et al., 2012). In gegenwärtigen akademschen Literatur zu dem Thema gibt es keine solchen Strategien. Die meisten Strategien unterscheiden nicht zwischen gewollten und ungewollten Retouren (Walsh et al., 2014). Das zweite Ziel dieser Dissertation besteht daher darin, die Basis für eine Strategie von Prognose und Intervention zu entwickeln, mit welcher Konsumverhalten mit hoher Retourenwahrscheinlichkeit im Vorfeld erkannt und rechtzeitig interveniert werden kann. In dieser Dissertation werden mehrere Prognosemodelle entwickelt, auf Basis welcher demonstriert wird, dass die Strategie, unter der Annahme moderat effektiver Interventionsstrategien, erhebliche Kosteneinsparungen mit sich bringt.

In today’s society a large portion of the worlds population get their news on electronicdevices. This opens up the possibility to enhance their reading experience bypersonalizing news for the readers based on their previous preferences. We have conductedan experiment to find out how accurately a Naïve Bayes classifier can selectarticles that a user might find interesting. Our experiments was done on two userswho read and classified 200 articles as interesting or not interesting. Those articleswere divided into four datasets with the sizes 50, 100, 150 and 200. We used a NaïveBayes classifier with 16 different settings configurations to classify the articles intotwo categories. From these experiments we could find several settings configurationsthat showed good results. One settings configuration was chosen as a good generalsetting for this kind of problem. We found that for datasets with a size larger than 50there were no significant increase in classification confidence.

Diagnosis is the first analysis step for uncovering the root cause of failure for a defective chip. It is a fast and non-destructive approach to preliminarily identify and locate possible defects in a failing chip. Despite many advances in diagnosis techniques, it is often the case, however, that resolution, i.e., the number of locations or candidates reported by diagnosis, exceeds the number of actual failing locations. To address this major challenge, a novel, machine-learning-based resolution improvement methodology named PADRE (Physically-Aware Diagnostic Resolution Enhancement) is described. PADRE uses easily-available tester and simulation data to extract features that uniquely characterize each candidate. PADRE applies machine learning to the features to identify candidates that correspond to the actual failure locations. Through various experiments, PADRE is shown to significantly improve resolution with virtually no negative impact on accuracy. Specifically, in simulation experiments, the number of defects that have perfect resolution is increased by 5x with little degradation of accuracy. An important investigation that typically follows diagnosis is Physical Failure Analysis (PFA), which can also provide information that is helpful for improving diagnosis. PADRE influences PFA within a novel, active learning (AL) based PFA selection approach. An active-learning based PADRE (AL PADRE) selects the most useful defects for PFA in order to improve diagnostic resolution. Experiments show AL PADRE can reach an accuracy of 90% with 60% less PFA, on average, compared to conventional defect selection for PFA. In addition, during the yield learning process, the failing mechanisms that lead to defective chips may change due to perturbations in the fabrication process. It is important for PADRE to perform robustly through the entire yield learning process. Therefore, additional techniques are developed to monitor the effectiveness of PADRE in real time, as well as to update PADRE efficiently and stably to cope with changing failure mechanisms.

Obstructive sleep apnoea (OSA) is a common disorder involving repeated cessations of breathing due to airway collapse, causing disruption of sleep cycles. The condition is under-diagnosed and the side effects are many and varied. Currently, the ‘gold standard’ diagnostic tool for OSA is a polysomnogram (PSG) which is carried out overnight in a hospital using multiple sensors. A PSG is expensive to set-up, run and analyse, and some subjects experience different sleep patterns due to the artificial conditions of the sleep laboratory. The aim of this thesis was to find a parsimonious and easy-to-collect set of signals (from the superset of signals recorded in sleep clinics) and other related information (such as demographics), and a set of automated methods that reliably determine which subjects are suitable for standard treatments, i.e. classify subjects requiring treatment (moderate OSA, severe OSA) from those not requiring treatment (normal, snorer, mild OSA), using a smartphone. Data were collected from 1354 subjects in the home using the Grey Flash polysomnographic recording device (Stowood Scientific Instruments, Oxford, UK). Analysis of the audio signal was initially performed using standard speech processing methods, where individual events were annotated and classified. The results achieved (accuracy (Ac) = 69.6%) using this approach were lower than those required for clinical acceptance. In all subsequent work in the thesis, subjects were classified from entire recordings rather than events. Multiscale entropy (MSE) was used to identify non-linear correlations in the audio data and quantify the irregularity of the data over many time scales. The inter-snore interval (ISI) was developed, motivated by clinical intuition. MSE and ISI were then applied to both actigraphy and photoplethysomgraphy (PPG) data, and different combinations of features were analysed. The features which displayed the highest predictive accuracy were derived from the PPG signal (Ac = 89.2%). This work demonstrated that, although audio- and actigraphy-based OSA screening is possible, to achieve clinically acceptable performance PPG remains an important key factor in diagnosis.

Globally distributed software teams are widespread throughout industry. But finding reliable methods that can properly assess a team's activities is a real challenge. Methods such as surveys and manual coding of activities are too time consuming and are often unreliable. Recent advances in information retrieval and linguistics, however, suggest that automated and/or semi-automated text classification algorithms could be an effective way of finding differences in the communication patterns among individuals and groups. Communication among group members is frequent and generates a significant amount of data. Thus having a web-based tool that can automatically analyze the communication patterns among global software teams could lead to a better understanding of group performance. The goal of this thesis, therefore, is to compare automatic and semi-automatic measures of communication and evaluate their effectiveness in classifying different types of group activities that occur within a global software development project. In order to achieve this goal, we developed a web-based component that can be used to help clean and classify communication activities. The component was then used to compare different automated text classification techniques on various group activities to determine their effectiveness in correctly classifying data from a global software development team project.

Two of the prime open-source environments available for machine/statistical learning in data mining and knowledge discovery are the software packages Weka and R which have emerged from the machine learning and statistics communities, respectively. To make the different sets of tools from both environments available in a single unified system, an R package RWeka is suggested which interfaces Weka's functionality to R. With only a thin layer of (mostly R) code, a set of general interface generators is provided which can set up interface functions with the usual "R look and feel", re-using Weka's standardized interface of learner classes (including classifiers, clusterers, associators, filters, loaders, savers, and stemmers) with associated methods. / Series: Research Report Series / Department of Statistics and Mathematics