The master thesis is carried out in a collaboration with a Swedish online grocery company. The goal of the thesis is to investigate if it is possible to explain the underlying factors that affect new customers to be retained. Because of the difficulties of defining churn and retention in non-contractual settings, most of the literature is focused on contractual and subscription settings. There are a limited number of studies when trying to predict customer churn in non-contractual businesses and even fewer studies that emphasize retention. This thesis aims to contribute to the field of retention in non-contractual business and also highlight the assumptions and drawbacks of churn-related task.  To achieve the goal of the thesis a literature review is carried out together with two statistical learning approaches; logistic regression model and extreme gradient boosting model. The results shows that it is possible to find the underlying factors that drive customers to be retained. The greatest drivers that could increase the probability of retaining new customers are the days between the first and second order, the second order value, and the total order value. / Examensarbetet är genomfört som ett samarbete med ett svenskt matvaruföretag på nätet. Målet med examensarbetet är att undersöka om det är möjligt att förklara de bakomliggande faktorer som påverkar nya kunder att stanna kvar som kunder. På grund av svårigheterna med att definiera kundbortfall och bibehållande av kunder i icke-kontraktuella affärer fokuserar den mesta av litteraturen på avtals- och prenumerationsmiljöer. Det finns ett begränsat antal studier där man försöker förutsäga kundbortfall i icke-kontraktuella verksamheter och ännu färre studier som fokuserar på bibehållande av kunder. Denna uppsats syftar till att bidra till området bibehållande av kunder i icke-kontraktuella affärer och även belysa antagandena och nackdelarna med analyser inom kundbortfall.  För att uppnå målet med avhandlingen genomförs en litteraturgenomgång tillsammans med två statistiska lärandemetoder; logistisk regressionsmodell och extreme gradient boosting model. Resultaten visar att det är fullt möjligt att hitta de bakomliggande faktorerna som driver kunderna att stanna kvar. De största drivkrafterna som kan öka sannolikheten för att kunder ska bibehållas är dagarna mellan första och andra ordern, andra ordervärdet och det totala ordervärdet.

The purpose of this study is to determine the effect of three improvement methods on nonparametric kernel regression estimators. The improvement methods are applied to the Nadaraya-Watson estimator with crossvalidation bandwidth selection, the Nadaraya-Watson estimator with plug-in bandwidth selection, the local linear estimator with plug-in bandwidth selection and a bias corrected nonparametric estimator proposed by Yao (2012). The di erent resulting regression estimates are evaluated by minimising a global discrepancy measure, i.e. the mean integrated squared error (MISE). In the machine learning context various improvement methods, in terms of the precision and accuracy of an estimator, exist. The rst two improvement methods introduced in this study are bootstrapped based. Bagging is an acronym for bootstrap aggregating and was introduced by Breiman (1996a) from a machine learning viewpoint and by Swanepoel (1988, 1990) in a functional context. Bagging is primarily a variance reduction tool, i.e. bagging is implemented to reduce the variance of an estimator and in this way improve the precision of the estimation process. Bagging is performed by drawing repetitive bootstrap samples from the original sample and generating multiple versions of an estimator. These replicates of the estimator are then used to obtain an aggregated estimator. Bragging stands for bootstrap robust aggregating. A robust estimator is obtained by using the sample median over the B bootstrap estimates instead of the sample mean as in bagging. The third improvement method aims to reduce the bias component of the estimator and is referred to as boosting. Boosting is a general method for improving the accuracy of any given learning algorithm. The method starts of with a sensible estimator and improves iteratively, based on its performance on a training dataset. Results and conclusions verifying existing literature are provided, as well as new results for the new methods. / MSc (Statistics), North-West University, Potchefstroom Campus, 2015

The purpose of this study is to determine the effect of three improvement methods on nonparametric kernel regression estimators. The improvement methods are applied to the Nadaraya-Watson estimator with crossvalidation bandwidth selection, the Nadaraya-Watson estimator with plug-in bandwidth selection, the local linear estimator with plug-in bandwidth selection and a bias corrected nonparametric estimator proposed by Yao (2012). The di erent resulting regression estimates are evaluated by minimising a global discrepancy measure, i.e. the mean integrated squared error (MISE). In the machine learning context various improvement methods, in terms of the precision and accuracy of an estimator, exist. The rst two improvement methods introduced in this study are bootstrapped based. Bagging is an acronym for bootstrap aggregating and was introduced by Breiman (1996a) from a machine learning viewpoint and by Swanepoel (1988, 1990) in a functional context. Bagging is primarily a variance reduction tool, i.e. bagging is implemented to reduce the variance of an estimator and in this way improve the precision of the estimation process. Bagging is performed by drawing repetitive bootstrap samples from the original sample and generating multiple versions of an estimator. These replicates of the estimator are then used to obtain an aggregated estimator. Bragging stands for bootstrap robust aggregating. A robust estimator is obtained by using the sample median over the B bootstrap estimates instead of the sample mean as in bagging. The third improvement method aims to reduce the bias component of the estimator and is referred to as boosting. Boosting is a general method for improving the accuracy of any given learning algorithm. The method starts of with a sensible estimator and improves iteratively, based on its performance on a training dataset. Results and conclusions verifying existing literature are provided, as well as new results for the new methods. / MSc (Statistics), North-West University, Potchefstroom Campus, 2015

The publication and mass appeal of Richard Thaler and Cass Sunstein's book Nudge: Improving Decisions about Health, Wealth and Happiness (Nudge), in 2008, illuminated behavioural economics, in the public and political domain. Nudging, a technique derived from behavioural economics, offers a fresh element to the long-time debate between paternalism and freedom, since proponents believe it can simultaneously preserve freedom of choice and serve as a means to influence behaviour. Unsurprisingly, in the decade or so since Nudge, private corporations and governments alike have shown great interest in the behavioural steering techniques derived from behavioural economics. This thesis explores the ethical implications and the various means by which governments and the private sector influence behaviour, specifically individual decision making. Since many of the methods overlap in purpose and practice, I make distinct three techniques: nudging, boosting and market advertising. These steering techniques range from transparent and educative to sub-conscious and manipulative methods; as such ethical justification for their employment varies. This thesis concludes by stating transparency as a condition for ethical behavioural influencing since non-transparent or covert methods do not uphold true freedom of choice, Furthermore, the implementation of non-transparent influences carries the potential for further violations of individual autonomy.

This study is a part of an experimental project at Statistics Sweden,which aims to, with the use of natural language processing and machine learning, predict Swedish businesses’ area of industry codes, based on their business descriptions. The response to predict consists of the most frequent 30 out of 88 main groups of Swedish standard industrial classification (SNI) codes that each represent a unique area of industry. The transformation from business description text to numerical features was done through the bag-of-words model. SNI codes are set when companies are founded, and due to the human factor, errors can occur. Using data from the Swedish Companies Registration Office, the purpose is to determine if the method of gradient boosting can provide high enough classification accuracy to automatically set the correct SNI codes that differ from the actual response. Today these corrections are made manually. The best gradient boosting model was able to correctly classify 52 percent of the observations, which is not considered high enough to implement automatic code correction into a production environment.

Cette thèse traite de la segmentation et du suivi de structures curvilinéaires. La méthodologie proposée est appliquée à la segmentation et au suivi des guide-fils durant les interventions d’angioplastie. Pendant ces opérations, les cardiologues s’assurent que le positionnement des différents outils est correct au moyen d’un système d’imagerie fluoroscopique temps-réel. Les images obtenues sont très bruitées et les guides sont, en conséquence, particulièrement difficiles à segmenter. Les contributions de cette thèse peuvent être regroupées en trois parties. La première est consacrée à la détection des guides, la seconde a leur segmentation et la dernière a leur suivi. La détection partielle des guide-fils est réalisée soit par la sélection d’un opérateur de filtrage approprié soit en utilisant des méthodes modernes d’apprentissage artificiel. Dans un premier temps, un système réalisant un Boosting asymétrique pour entraîner un détecteur de guides est présenté. Par la suite, une méthode renforçant la réponse d’un filtre orientable au moyen d’une variante efficace de vote tensoriel est décrite. Dans la seconde partie, une approche ascendante est proposée, qui consiste à regrouper des points sélectionnés par le détecteur de fil, à extraire des primitives des agrégats obtenus et a les lier. Deux procédures locales de regroupement des points sont étudiées : une reposant sur un clustering de graphe non supervisé suivi d’une extraction de segments de droites ; et l’autre reposant sur un modèle graphique puis une extraction d’axe central. Par la suite, deux méthodes de liaison des primitives sont étudiées : la première repose sur une approche de programmation linéaire, et la seconde sur une heuristique de recherche locale. Dans la dernière partie, des méthodes de recalage sont utilisées pour améliorer la segmentation et pour suivre les fils. Le suivi propos´e couple un suivi iconique avec un suivi géométrique contenant un modèle prédictif. Cette méthode utilise un modèle graphique déterminant à la fois une position du guide-fil (segmentation) et des correspondances (tracking). La solution optimale de ce modèle graphique décrit simultanément les déplacements du guide-fil et les appariements entre points d’intérêt qui en sont extraits, fournissant ainsi une estimation robuste des déformations du fil par rapport aux grands déplacements et au bruit. / This thesis addresses the segmentation and the tracking of thin curvilinear structures. The proposed methodology is applied to the delineation and the tracking of the guide-wires that are used during cardiac angioplasty. During these interventions, cardiologists assess the displacement of the different devices with a real-time fluoroscopic imaging system. The obtained images are very noisy and, as a result, guide-wires are particularly challenging to segment and track. The contributions of this thesis can be grouped into three parts. The first part is devoted to the detection of the guide-wires, the second part addresses their segmentation and the last part focuses on their spatio-temporal tracking. Partial detection of guide-wires is addressed either through the selection of appropriate filter operators or using modern machine learning methods. First, a learning framework using an asymmetric Boosting algorithm for training a guidewire detector is presented. A second method enhancing the output of a steerable filter by using an efficient tensor voting variant is then described. In the second part, a bottom-up method is proposed, that consists in grouping points selected by the wire detector, in extracting primitives from these aggregates and in linking these primitives together. Two local grouping procedures are investigated: one based on unsupervised graph-based clustering followed by a linesegment extraction and one based on a graphical model formulation followed by a graph-based centerline extraction. Subsequently, two variants of linking methods are investigated: one is based on integer programming and one on a local search heuristic. In the last part, registration methods are exploited for improving the segmentation via an image fusion method and then for tracking the wires. This latter is performed by a graph-based iconic tracking method coupled with a graphbased geometric tracking that encodes to certain extend a predictive model. This method uses a coupled graphical model that seeks both optimal position (segmentation) and spatio-temporal correspondences (tracking). The optimal solution of this graphical model simultaneously determines the guide-wire displacements and matches the landmarks that are extracted along it, what provides a robust estimation of the wire deformations with respect to large motion and noise.

The usual convention for assigning a treatment to an individual is a "one-size fits all" rule that is based on broad spectrum trends. Heterogeneity within and between subjects and improvements in scientific research convey the need for more effective treatment assignment strategies. Precisionized treatment (PT) offers an alternative to the traditional treatment assignment approach by making treatment decisions based on one or more covariates pertaining to an individual. We investigate two methods to inform PT rules: the Maximum Likelihood Estimation (MLE) method and the Boosting method. We apply these methods in the context of a crossover study design with a continuous outcome variable, one continuous covariate, and two intervention options. We explore the methods via extensive simulation studies and apply them to a data set from a study of safety warnings in passenger vehicles. We evaluate the performance of the estimated PT rules based on the improvement in mean response (RMD), the percent of correct treatment assignments (PCC), and the accuracy of estimating the location of the crossing point (MSE((x_c )). We also define a new metric that we call the percent of anomalies (PA). We characterize the potential benefit of using PT by relating it to the strength of interaction, the location of the crossing point, and the within-person intraclass correlation (ICC). We also explore the effects of sample size and overall variance along with the methods’ robustness to violations of model assumptions. We investigate the performance of the Boosting method under the standard weight and two alternative weighting schemes. Our investigation indicated the largest potential benefit of implementing a PT approach was when the crossover point was near the median, the strength of interaction was large, and the ICC was high. When a PT rule is used to assign treatments instead of a one-size fits all rule, an approximate 10-30% improvement in mean outcome can be gained. The MLE and Boosting method performed comparably across most of the simulation scenarios, yet in our data example, it appeared there may be an empirical benefit of the Boosting method over the MLE method. Under a distribution misspecification, the difference in performance between the methods was minor; however, when the functional form of the model was misspecified, we began to see improvement of the Boosting method over the MLE method. In the simulation conditions we considered, the weighting scheme used in the Boosting method did not markedly impact performance. Using data to develop PT rules can lead to an improvement in outcome over the standard approach of assigning treatments. We found that in a variety of scenarios, there was little added benefit to utilizing the more complex iterative Boosting procedure compared to the relatively straightforward MLE method when developing the PT rules. The results from our investigations could be used to optimize treatment recommendations for participants in future studies.

La génération massive de données, nous avons de plus en plus de données issues de différentes sources d’informations ayant des propriétés hétérogènes. Il est donc important de prendre en compte ces représentations ou vues des données. Ce problème d'apprentissage automatique est appelé apprentissage multivue. Il est utile dans de nombreux domaines d’applications, par exemple en imagerie médicale, nous pouvons représenter le cerveau humains via des IRM, t-fMRI, EEG, etc. Dans cette cette thèse, nous nous concentrons sur l’apprentissage multivue supervisé, où l’apprentissage multivue est une combinaison de différents modèles de classifications ou de vues. Par conséquent, selon notre point de vue, il est intéressant d’aborder la question de l’apprentissage à vues multiples dans le cadre PAC-Bayésien. C’est un outil issu de la théorie de l’apprentissage statistique étudiant les modèles s’exprimant comme des votes de majorité. Un des avantages est qu’elle permet de prendre en considération le compromis entre précision et diversité des votants, au cœur des problématiques liées à l’apprentissage multivue. La première contribution de cette thèse étend la théorie PAC-Bayésienne classique (avec une seule vue) à l’apprentissage multivue (avec au moins deux vues). Pour ce faire, nous définissons une hiérarchie de votants à deux niveaux: les classifieurs spécifiques à la vue et les vues elles-mêmes. Sur la base de cette stratégie, nous avons dérivé des bornes en généralisation PAC-Bayésiennes (probabilistes et non-probabilistes) pour l’apprentissage multivue. D'un point de vue pratique, nous avons conçu deux algorithmes d'apprentissage multivues basés sur notre stratégie PAC-Bayésienne à deux niveaux. Le premier algorithme appelé PB-MVBoost est un algorithme itératif qui apprend les poids sur les vues en contrôlant le compromis entre la précision et la diversité des vues. Le second est une approche de fusion tardive où les prédictions des classifieurs spécifiques aux vues sont combinées via l’algorithme PAC-Bayésien CqBoost proposé par Roy et al. Enfin, nous montrons que la minimisation des erreurs pour le vote de majorité multivue est équivalente à la minimisation de divergences de Bregman. De ce constat, nous proposons un algorithme appelé MωMvC2 pour apprendre un vote de majorité multivue. / With tremendous generation of data, we have data collected from different information sources having heterogeneous properties, thus it is important to consider these representations or views of the data. This problem of machine learning is referred as multiview learning. It has many applications for e.g. in medical imaging, we can represent human brain with different set of features for example MRI, t-fMRI, EEG, etc. In this thesis, we focus on supervised multiview learning, where we see multiview learning as combination of different view-specific classifiers or views. Therefore, according to our point of view, it is interesting to tackle multiview learning issue through PAC-Bayesian framework. It is a tool derived from statistical learning theory studying models expressed as majority votes. One of the advantages of PAC-Bayesian theory is that it allows to directly capture the trade-off between accuracy and diversity between voters, which is important for multiview learning. The first contribution of this thesis is extending the classical PAC-Bayesian theory (with a single view) to multiview learning (with more than two views). To do this, we considered a two-level hierarchy of distributions over the view-specific voters and the views. Based on this strategy, we derived PAC-Bayesian generalization bounds (both probabilistic and expected risk bounds) for multiview learning. From practical point of view, we designed two multiview learning algorithms based on our two-level PAC-Bayesian strategy. The first algorithm is a one-step boosting based multiview learning algorithm called as PB-MVBoost. It iteratively learns the weights over the views by optimizing the multiview C-Bound which controls the trade-off between the accuracy and the diversity between the views. The second algorithm is based on late fusion approach where we combine the predictions of view-specific classifiers using the PAC-Bayesian algorithm CqBoost proposed by Roy et al. Finally, we show that minimization of classification error for multiview weighted majority vote is equivalent to the minimization of Bregman divergences. This allowed us to derive a parallel update optimization algorithm (referred as MωMvC2) to learn our multiview weighted majority vote.

Modern information technology allows information to be collected at a far greater rate than ever before. So fast, in fact, that the main problem is making sense of it all. Machine learning offers promise of a solution, but the field mainly focusses on achieving high accuracy when data supply is limited. While this has created sophisticated classification algorithms, many do not cope with increasing data set sizes. When the data set sizes get to a point where they could be considered to represent a continuous supply, or data stream, then incremental classification algorithms are required. In this setting, the effectiveness of an algorithm cannot simply be assessed by accuracy alone. Consideration needs to be given to the memory available to the algorithm and the speed at which data is processed in terms of both the time taken to predict the class of a new data sample and the time taken to include this sample in an incrementally updated classification model. The Hoeffding tree algorithm is a state-of-the-art method for inducing decision trees from data streams. The aim of this thesis is to improve this algorithm. To measure improvement, a comprehensive framework for evaluating the performance of data stream algorithms is developed. Within the framework memory size is fixed in order to simulate realistic application scenarios. In order to simulate continuous operation, classes of synthetic data are generated providing an evaluation on a large scale. Improvements to many aspects of the Hoeffding tree algorithm are demonstrated. First, a number of methods for handling continuous numeric features are compared. Second, tree prediction strategy is investigated to evaluate the utility of various methods. Finally, the possibility of improving accuracy using ensemble methods is explored. The experimental results provide meaningful comparisons of accuracy and processing speeds between different modifications of the Hoeffding tree algorithm under various memory limits. The study on numeric attributes demonstrates that sacrificing accuracy for space at the local level often results in improved global accuracy. The prediction strategy shown to perform best adaptively chooses between standard majority class and Naive Bayes prediction in the leaves. The ensemble method investigation shows that combining trees can be worthwhile, but only when sufficient memory is available, and improvement is less likely than in traditional machine learning. In particular, issues are encountered when applying the popular boosting method to streams.

Ensemble techniques have recently been used to enhance the performance of machine learning methods. However, current ensemble techniques for classification require both positive and negative data to produce a result that is both meaningful and useful. Negative data is, however, sometimes difficult, expensive or impossible to access. In this thesis a learning framework is described that has a very close relationship to boosting. Within this framework a method is described which bears remarkable similarities to boosting stumps and that does not rely on negative examples. This is surprising since learning from positive-only data has traditionally been difficult. An empirical methodology is described and deployed for testing positive-only learning systems using commonly available multiclass datasets to compare these learning systems with each other and with multiclass learning systems. Empirical results show that our positive-only boosting-like method learns, using stumps as a base learner and from positive data only, successfully, and in the process does not pay too heavy a price in accuracy compared to learners that have access to both positive and negative data. We also describe methods of using positive-only learners on multiclass learning tasks and vice versa and empirically demonstrate the superiority of our method of learning in a boosting-like fashion from positive-only data over a traditional multiclass learner converted to learn from positive-only data. Finally we examine some alternative frameworks, such as when additional unlabelled training examples are given. Some theoretical justifications of the results and methods are also provided.