The thesis proposes a method for automated model selection using a logistic regression model in the context of prospectivity modeling, i.e. the exploration of minearlisations. This kind of data is characterized by a rare positive event and a large dataset. We adapted and combined the two statistical measures Wald statistic and Bayes' information criterion making it suitable for the processing of large data and a high number of variables that emerge in the nonlinear setting of logistic regression. The obtained models of our suggested method are parsimonious allowing for an interpretation and information gain. The advantages of our method are shown by comparing it to another model selection method and to arti cial neural networks on several datasets. Furthermore we introduced a possibility to induce spatial dependencies which are important in such geological settings.

La théorie des jeux à champ moyen fut introduite en 2006 par Jean-Michel Lasry et Pierre-Louis Lions. Elle permet l'étude de la théorie des jeux dans certaines configurations où le nombre de joueurs est trop grand pour espérer une résolution pratique. Nous étudions la théorie des jeux à champ moyen sur les graphes en nous appuyant sur les travaux d'Olivier Guéant que nous étendrons à des formes plus générales d'Hilbertien. Nous étudierons aussi les liens qui existent entres les K-moyennes et les jeux à champ moyen ce qui permettra en principe de proposer de nouveaux algorithmes pour les K-moyennes grâce aux techniques de résolution numérique propres aux jeux à champ moyen. Enfin nous étudierons un jeu à champ moyen à savoir le problème "d'heure de début d'une réunion" en l'étendant à des situations où les agents peuvent choisir entre deux réunions. Nous étudierons de manière analytique et numérique l'existence et la multiplicité des solutions de ce problème. / The mean field game theory was introduced in 2006 by Jean-Michel Lasry and Pierre-Louis Lions. It allows us to study the game theory in some situations where the number of players is too high to be able to be solved in practice. We will study the mean field game theory on graphs by learning from the studies of Oliver Guéant which we will extend to more generalized forms of Hilbertian. We will also study the links between the K-means and the mean field game theory. In principle, this will offer us new algorithms for solving the K-means thanks to the techniques of numerical resolutions of the mean field games. Findly, we will study a mean field game called the "starting time of a meeting". We will extend it to situations where the players can choose between two meetings. We will study analytically and numerically the existence and multiplicity of the solutions to this problem.

Email communication is valuable for any modern company, since it offers an easy mean for spreading important information or advertising new products, features or offers and much more. To be able to identify which customers that would be interested in certain information would make it possible to significantly improve a company's email communication and as such avoiding that customers start ignoring messages and creating unnecessary badwill. This thesis focuses on trying to target customers by applying statistical learning methods to historical data provided by the music streaming company Spotify. An important aspect was the high-dimensionality of the data, creating certain demands on the applied methods. A binary classification model was created, where the target was whether a customer will open the email or not. Two approaches were used for trying to target the costumers, logistic regression, both with and without regularization, and random forest classifier, for their ability to handle the high-dimensionality of the data. Performance accuracy of the suggested models were then evaluated on both a training set and a test set using statistical validation methods, such as cross-validation, ROC curves and lift charts. The models were studied under both large-sample and high-dimensional scenarios. The high-dimensional scenario represents when the number of observations, N, is of the same order as the number of features, p and the large sample scenario represents when N ≫ p. Lasso-based variable selection was performed for both these scenarios, to study the informative value of the features. This study demonstrates that it is possible to greatly improve the opening rate of emails by targeting users, even in the high dimensional scenario. The results show that increasing the amount of training data over a thousand fold will only improve the performance marginally. Rather efficient customer targeting can be achieved by using a few highly informative variables selected by the Lasso regularization. / Företag kan använda e-mejl för att på ett enkelt sätt sprida viktig information, göra reklam för nya produkter eller erbjudanden och mycket mer, men för många e-mejl kan göra att kunder slutar intressera sig för innehållet, genererar badwill och omöjliggöra framtida kommunikation. Att kunna urskilja vilka kunder som är intresserade av det specifika innehållet skulle vara en möjlighet att signifikant förbättra ett företags användning av e-mejl som kommunikationskanal. Denna studie fokuserar på att urskilja kunder med hjälp av statistisk inlärning applicerad på historisk data tillhandahållen av musikstreaming-företaget Spotify. En binärklassificeringsmodell valdes, där responsvariabeln beskrev huruvida kunden öppnade e-mejlet eller inte. Två olika metoder användes för att försöka identifiera de kunder som troligtvis skulle öppna e-mejlen, logistisk regression, både med och utan regularisering, samt random forest klassificerare, tack vare deras förmåga att hantera högdimensionella data. Metoderna blev sedan utvärderade på både ett träningsset och ett testset, med hjälp av flera olika statistiska valideringsmetoder så som korsvalidering och ROC kurvor. Modellerna studerades under både scenarios med stora stickprov och högdimensionella data. Där scenarion med högdimensionella data representeras av att antalet observationer, N, är av liknande storlek som antalet förklarande variabler, p, och scenarion med stora stickprov representeras av att N ≫ p. Lasso-baserad variabelselektion utfördes för båda dessa scenarion för att studera informationsvärdet av förklaringsvariablerna. Denna studie visar att det är möjligt att signifikant förbättra öppningsfrekvensen av e-mejl genom att selektera kunder, även när man endast använder små mängder av data. Resultaten visar att en enorm ökning i antalet träningsobservationer endast kommer förbättra modellernas förmåga att urskilja kunder marginellt.

Today, digitalization is a key factor for businesses to enhance growth and gain advantages and insight in their operations. Both in planning operations and understanding customers the digitalization processes today have key roles, and companies are spending more and more resources in this fields to gain critical insights and enhance growth. The fast-food industry is no exception where restaurants need to be highly flexible and agile in their work. With this, there exists an immense demand for knowledge and insights to help restaurants plan their daily operations and there is a great need for organizations to continuously adapt new technological solutions into their existing processes. Well implemented Machine Learning solutions in combination with feature engineering are likely to bring value into the existing processes. Sales forecasting, which is the main field of study in this thesis work, has a vital role in planning of fast food restaurant's operations, both for budgeting purposes, but also for staffing purposes. The word fast food describes itself. With this comes a commitment to provide high quality food and rapid service to the customers. Understaffing can risk violating either quality of the food or service while overstaffing leads to low overall productivity. Generating highly reliable sales forecasts are thus vital to maximize profits and minimize operational risk. SARIMA, XGBoost and Random Forest were evaluated on training data consisting of sales numbers, business hours and categorical variables describing date and month. These models worked as base learners where sales predictions from a specific dataset were used as training data for a Support Vector Regression model (SVR). A stacking approach to this type of project shows sufficient results with a significant gain in prediction accuracy for all investigated restaurants on a 6-week aggregated timeline compared to the existing solution. / Digitalisering har idag en nyckelroll för att skapa tillväxt och insikter för företag, dessa insikter ger fördelar både inom planering och i förståelsen om deras kunder. Det här är ett område som företag lägger mer och mer resurser på för att skapa större förståelse om sin verksamhet och på så sätt öka tillväxten. Snabbmatsindustrin är inget undantag då restauranger behöver en hög grad av flexibilitet i sina arbetssätt för att möta kundbehovet. Det här skapar en stor efterfrågan av kunskap och insikter för att hjälpa dem i planeringen av deras dagliga arbete och det finns ett stort behov från företagen att kontinuerligt implementera nya tekniska lösningar i befintliga processer. Med väl implementerade maskininlärningslösningar i kombination med att skapa mer informativa variabler från befintlig data kan aktörer skapa mervärde till redan existerande processer. Försäljningsprognostisering, som är huvudområdet för den här studien, har en viktig roll för verksamhetsplaneringen inom snabbmatsindustrin, både inom budgetering och bemanning. Namnet snabbmat beskriver sig själv, med det följer ett löfte gentemot kunden att tillhandahålla hög kvalitet på maten samt att kunna tillhandahålla snabb service. Underbemanning kan riskera att bryta någon av dessa löften, antingen i undermålig kvalitet på maten eller att inte kunna leverera snabb service. Överbemanning riskerar i stället att leda till ineffektivitet i användandet av resurser. Att generera högst tillförlitliga prognoser är därför avgörande för att kunna maximera vinsten och minimera operativ risk. SARIMA, XGBoost och Random Forest utvärderades på ett träningsset bestående av försäljningssiffror, timme på dygnet och kategoriska variabler som beskriver dag och månad. Dessa modeller fungerar som basmodeller vars prediktioner från ett specifikt testset används som träningsdata till en Stödvektorsreggresionsmodell (SVR). Att använda stapling av maskininlärningsmodeller till den här typen av problem visade tillfredställande resultat där det påvisades en signifikant förbättring i prediktionssäkerhet under en 6 veckors aggregerad period gentemot den redan existerande modellen.

This thesis focuses on two topics, namely statistical learning and the prediction of key performance indicators in the performance evaluation of a storage engine. The part on statistical learning presents a novel algorithm adjusting the sampling size for the Monte Carlo approximation of the function to be minimized, allowing a reduction of the true function at a given probability and this, at a lower numerical cost. The sampling strategy is embedded in a trust-region algorithm, using the Fisher Information matrix, also called BHHH approximation, to approximate the Hessian matrix. The sampling strategy is tested on a logit model generated from synthetic data. Numerical results exhibit a significant reduction in the time required to optimize the model when an adequate smoothing is applied to the function. The key performance indicator prediction part describes a novel strategy to select better settings for RocksDB that optimize its throughput, using the log files to analyze and identify suboptimal parameters, opening the possibility to greatly accelerate modern storage engine tuning. / Ce mémoire s’intéresse à deux sujets, un relié à l’apprentisage statistique et le second à la prédiction d’indicateurs de performance dans un système de stockage de type clé-valeur. La partie sur l’apprentissage statistique développe un algorithme ajustant la taille d’échantillonnage pour l’approximation Monte Carlo de la fonction à minimiser, permettant une réduction de la véritable fonction avec une probabilité donnée, et ce à un coût numérique moindre. La stratégie d’échantillonnage est développée dans un contexte de région de confiance en utilisant la matrice d’information de Fisher, aussi appelée approximation BHHH de la matrice hessienne. La stratégie d’échantillonnage est testée sur un modèle logit généré à partir de données synthétiques suivant le même modèle. Les résultats numériques montrent une réduction siginificative du temps requis pour optimiser le modèle lorsqu’un lissage adéquat est appliqué. La partie de prédiction d’indicateurs de performance décrit une nouvelle approche pour optimiser la vitesse maximale d’insertion de paire clé-valeur dans le système de stockage RocksDB. Les fichiers journaux sont utilisés pour identifier les paramètres sous-optimaux du système et accélérer la recherche de paramètres optimaux.

Une exploitation intelligente des données qui circulent sur les réseaux pourrait entraîner une amélioration de la qualité d'expérience (QoE) des utilisateurs. Les techniques d'apprentissage automatique offrent des fonctionnalités multiples, ce qui permet d’optimiser l'utilisation des ressources réseau.Dans cette thèse, deux contextes d’application sont étudiés : les réseaux de capteurs sans fil (WSNs) et les réseaux de contenus (CDNs). Dans les WSNs, il s’agit de prédire la qualité des liens sans fil afin d’améliorer la qualité des routes et donc d’augmenter le taux de remise des paquets ce qui améliore la qualité de service offerte à l’utilisateur. Dans les CDNs, il s’agit de prédire la popularité des contenus vidéo afin de mettre en cache les contenus les plus populaires, au plus près des utilisateurs qui les demandent. Ceci contribue à réduire la latence pour satisfaire les requêtes des utilisateurs.Dans ce travail, nous avons orchestré des techniques d’apprentissage issues de deux domaines différents, à savoir les statistiques et le Machine Learning. Chaque technique est représentée par un expert dont les paramètres sont réglés suite à une analyse hors-ligne. Chaque expert est chargé de prédire la prochaine valeur de la métrique. Vu la variété des experts retenus et comme aucun d’entre eux ne domine toujours tous les autres, un deuxième niveau d’expertise est nécessaire pour fournir la meilleure prédiction. Ce deuxième niveau est représenté par un expert particulier, appelé forecaster. Le forecaster est chargé de fournir des prédictions à partir des prédictions fournies par un sous ensemble des meilleurs experts.Plusieurs méthodes d’identification de ce sous ensemble sont étudiées. Elles dépendent de la fonction de perte utilisée pour évaluer les prédictions des experts et du nombre k, représentant les k meilleurs experts. Les tâches d’apprentissage et de prédiction sont effectuées en-ligne sur des data sets réels issus d’un WSN déployé à Stanford et de YouTube pour le CDN. La méthodologie adoptée dans cette thèse s’applique à la prédiction de la prochaine valeur d’une série temporelle.Plus précisément, nous montrons comment dans le contexte WSN, la qualité des liens peut être évaluée par le Link Quality Indicator (LQI) et comment les experts Single Exponential Smoothing (SES) et Average Moving Window (AMW) peuvent prédire la prochaine valeur de LQI. Ces experts réagissent rapidement aux changements des valeurs LQI que ce soit lors d’une brusque baisse de la qualité du lien ou au contraire lors d’une forte augmentation de la qualité. Nous proposons deux forecasters, Exponential Weighted Average (EWA) et Best Expert (BE), et fournissons la combinaison Expert-Forecaster permettant de fournir la meilleure prédiction.Dans le contexte des CDNs, nous évaluons la popularité de chaque contenu vidéo par le nombre journalier de requêtes. Nous utilisons à la fois des experts statistiques (ARMA) et des experts issus du Machine Learning (DES, régression polynômiale). Nous introduisons également des forecasters qui diffèrent par rapport à l’horizon des observations utilisées pour la prédiction, la fonction de perte et le nombre d’experts utilisés. Ces prédictions permettent de décider quels contenus seront placés dans les caches proches des utilisateurs. L’efficacité de la technique de caching basée sur la prédiction de la popularité est évaluée en termes de hit ratio et d’update ratio. Nous mettons en évidence les apports de cette technique de caching par rapport à un algorithme de caching classique, Least Frequently Used (LFU).Cette thèse se termine par des recommandations concernant l’utilisation des techniques d’apprentissage en ligne et hors-ligne pour les réseaux (WSN, CDN). Au niveau des perspectives, nous proposons différentes applications où l’utilisation de ces techniques permettrait d’améliorer la qualité d’expérience des utilisateurs mobiles ou des utilisateurs des réseaux IoT. / An intelligent exploitation of data carried on telecom networks could lead to a very significant improvement in the quality of experience (QoE) for the users. Machine Learning techniques offer multiple operating, which can help optimize the utilization of network resources.In this thesis, two contexts of application of the learning techniques are studied: Wireless Sensor Networks (WSNs) and Content Delivery Networks (CDNs). In WSNs, the question is how to predict the quality of the wireless links in order to improve the quality of the routes and thus increase the packet delivery rate, which enhances the quality of service offered to the user. In CDNs, it is a matter of predicting the popularity of videos in order to cache the most popular ones as close as possible to the users who request them, thereby reducing latency to fulfill user requests.In this work, we have drawn upon learning techniques from two different domains, namely statistics and Machine Learning. Each learning technique is represented by an expert whose parameters are tuned after an off-line analysis. Each expert is responsible for predicting the next metric value (i.e. popularity for videos in CDNs, quality of the wireless link for WSNs). The accuracy of the prediction is evaluated by a loss function, which must be minimized. Given the variety of experts selected, and since none of them always takes precedence over all the others, a second level of expertise is needed to provide the best prediction (the one that is the closest to the real value and thus minimizes a loss function). This second level is represented by a special expert, called a forecaster. The forecaster provides predictions based on values predicted by a subset of the best experts.Several methods are studied to identify this subset of best experts. They are based on the loss functions used to evaluate the experts' predictions and the value k, representing the k best experts. The learning and prediction tasks are performed on-line on real data sets from a real WSN deployed at Stanford, and from YouTube for the CDN. The methodology adopted in this thesis is applied to predicting the next value in a series of values.More precisely, we show how the quality of the links can be evaluated by the Link Quality Indicator (LQI) in the WSN context and how the Single Exponential Smoothing (SES) and Average Moving Window (AMW) experts can predict the next LQI value. These experts react quickly to changes in LQI values, whether it be a sudden drop in the quality of the link or a sharp increase in quality. We propose two forecasters, Exponential Weighted Average (EWA) and Best Expert (BE), as well as the Expert-Forecaster combination to provide better predictions.In the context of CDNs, we evaluate the popularity of each video by the number of requests for this video per day. We use both statistical experts (ARMA) and experts from the Machine Learning domain (e.g. DES, polynomial regression). These experts are evaluated according to different loss functions. We also introduce forecasters that differ in terms of the observation horizon used for prediction, loss function and number of experts selected for predictions. These predictions help decide which videos will be placed in the caches close to the users. The efficiency of the caching technique based on popularity prediction is evaluated in terms of hit rate and update rate. We highlight the contributions of this caching technique compared to a classical caching algorithm, Least Frequently Used (LFU).This thesis ends with recommendations for the use of online and offline learning techniques for networks (WSN, CDN). As perspectives, we propose different applications where the use of these techniques would improve the quality of experience for mobile users (cellular networks) or users of IoT (Internet of Things) networks, based, for instance, on Time Slotted Channel Hopping (TSCH).

La Génération Automatique de Texte (GAT) est le champ de recherche de la linguistique informatique qui étudie la possibilité d’attribuer à une machine la faculté de produire du texte intelligible. Dans ce mémoire, nous présentons une proposition de système de GAT reposant exclusivement sur des méthodes statistiques. Son originalité est d’exploiter un corpus en tant que ressource de formation de phrases. Cette méthode offre plusieurs avantages : elle simplifie l’implémentation d’un système de GAT en plusieurs langues et améliore les capacités d’adaptations d’un système de génération à un domaine sémantique particulier. La production, d’après un corpus d’apprentissage, des modèles de phrases finement étiquetées requises par notre générateur de texte nous a conduit à mener des recherches approfondies dans le domaine de l’extraction d’information et de la classification. Nous décrivons le système d’étiquetage et de classification de contenus encyclopédique mis au point à cette fin. Dans les étapes finales du processus de génération, les modèles de phrases sont exploités par un module de génération de texte multilingue. Ce module exploite des algorithmes de recherche d’information pour extraire du modèle une phrase pré-existante, utilisable en tant que support sémantique et syntaxique de l’intention à communiquer. Plusieurs méthodes sont proposées pour générer une phrase, choisies en fonction de la complexité du contenu sémantique à exprimer. Nous présentons notamment parmi ces méthodes une proposition originale de génération de phrases complexes par agrégation de proto-phrases de type Sujet, Verbe, Objet. Nous envisageons dans nos conclusions que cette méthode particulière de génération puisse ouvrir des voies d’investigations prometteuses sur la nature du processus de formation de phrases / Natural Language Generation (NLG) is the natural language processing task of generating natural language from a machine representation system. In this thesis report, we present an architecture of NLG system relying on statistical methods. The originality of our proposition is its ability to use a corpus as a learning resource for sentences production. This method offers several advantages : it simplifies the implementation and design of a multilingual NLG system, capable of sentence production of the same meaning in several languages. Our method also improves the adaptability of a NLG system to a particular semantic field. In our proposal, sentence generation is achieved trough the use of sentence models, obtained from a training corpus. Extracted sentences are abstracted by a labelling step obtained from various information extraction and text mining methods like named entity recognition, co-reference resolution, semantic labelling and part of speech tagging. The sentence generation process is achieved by a sentence realisation module. This module provide an adapted sentence model to fit a communicative intent, and then transform this model to generate a new sentence. Two methods are proposed to transform a sentence model into a generated sentence, according to the semantic content to express. In this document, we describe the complete labelling system applied to encyclopaedic content to obtain the sentence models. Then we present two models of sentence generation. The first generation model substitute the semantic content to an original sentence content. The second model is used to find numerous proto-sentences, structured as Subject, Verb, Object, able to fit by part a whole communicative intent, and then aggregate all the selected proto-sentences into a more complex one. Our experiments of sentence generation with various configurations of our system have shown that this new approach of NLG have an interesting potential

GPS tracking data are widely used to understand human travel behavior and to evaluate the impact of travel. A major advantage with the usage of GPS tracking devices for collecting data is that it enables the researcher to collect large amounts of highly accurate and detailed human mobility data. However, unlabeled GPS tracking data does not easily lend itself to detecting transportation mode and this has given rise to a range of methods and algorithms for this purpose. The algorithms used vary in design and functionality, from defining specific rules to advanced machine learning algorithms. There is however no previous comprehensive review of these algorithms and this thesis aims to identify their essential features and methods and to develop and demonstrate a method for the detection of transport mode in GPS tracking data. To do this, it is necessary to have a detailed description of the particular journey undertaken by an individual. Therefore, as part of the investigation, a microdata analytic approach is applied to the problem areas, including the stages of data collection, data processing, analyzing the data, and decision making. In order to fill the research gap, Paper I consists of a systematic literature review of the methods and essential features used for detecting the transport mode in unlabeled GPS tracking data. Selected empirical studies were categorized into rule-based methods, statistical methods, and machine learning methods. The evaluation shows that machine learning algorithms are the most common. In the evaluation, I compared the methods previously used, extracted features, types of dataset, and model accuracy of transport mode detection. The results show that there is no standard method used in transport mode detection. In the light of these results, I propose in Paper II a stepwise methodology to detect five transport modes taking advantage of the unlabeled GPS data by first using an unsupervised algorithm to detect the five transport modes. A GIS multi-criteria process was applied to label part of the dataset. The performance of the five supervised algorithms was evaluated by applying them to different portions of the labeled dataset. The results show that stepwise methodology can achieve high accuracy in detecting the transport mode by labeling only 10% of the data from the entire dataset.  For the future, one interesting area to explore would be the application of the stepwise methodology to a balanced and larger dataset. A semi-supervised deep-learning approach is suggested for development in transport mode detection, since this method can detect transport modes with only small amounts of labeled data. Thus, the stepwise methodology can be improved upon for further studies.

Data-driven prediction of battery health has gained increased attention over the past couple of years, in both academia and industry. Accurate early-stage predictions of battery performance would create new opportunities regarding production and use. Using data from only the first 100 cycles, in a data set of 124 cells where lifetimes span between 150 and 2300 cycles, this work combines parametric linear models with non-parametric Gaussian process regression to achieve cycle lifetime predictions with an overall accuracy of 8.8% mean error. This work presents a relevant contribution to current research as this combination of methods is previously unseen when regressing battery lifetime on a high dimensional feature space. The study and the results presented further show that Gaussian process regression can serve as a valuable contributor in future data-driven implementations of battery health predictions. / Datadriven prediktion av batterihälsa har fått ökad uppmärksamhet under de senaste åren, både inom akademin och industrin. Precisa prediktioner i tidigt stadium av batteriprestanda skulle kunna skapa nya möjligheter för produktion och användning. Genom att använda data från endast de första 100 cyklerna, i en datamängd med 124 celler där livslängden sträcker sig mellan 150 och 2300 cykler, kombinerar denna uppsats parametriska linjära modeller med ickeparametrisk Gaussisk processregression för att uppnå livstidsprediktioner med en genomsnittlig noggrannhet om 8.8% fel. Studien utgör ett relevant bidrag till den aktuella forskningen eftersom den använda kombinationen av metoder inte tidigare utnyttjats för regression av batterilivslängd med ett högdimensionellt variabelrum. Studien och de erhållna resultaten visar att regression med hjälp av Gaussiska processer kan bidra i framtida datadrivna implementeringar av prediktion för batterihälsa.

Data gathering is a constant in human history with ever increasing amounts in quantity and dimensionality. To get a feel for the data, make it interpretable, or find underlying laws it is necessary to fit a function to the finite and possibly noisy data. In this thesis we focus on a method achieving this, namely least squares approximation. Its discovery dates back to around 1800 and it has since then proven to be an indispensable tool which is efficient and has the capability to achieve optimal error when used right. Crucial for the least squares method are the ansatz functions and the sampling points. To discuss them, we gather tools from probability theory, frame subsampling, and $L_2$-Marcinkiewicz-Zygmund inequalities. With that we give results in the worst-case or minmax setting, when a set of points is sought for approximating a class of functions, which we model as a generic reproducing kernel Hilbert space. Further, we give error bounds in the statistical learning setting for approximating individual functions from possibly noisy samples. Here, we include the covariate-shift setting as a subfield of transfer learning. In a natural way a parameter choice question arises for balancing over- and underfitting effect. We tackle this by using the cross-validation score, for which we show a fast way of computing as well as prove the goodness thereof.:1 Introduction 2 Least squares approximation 3 Reproducing kernel Hilbert spaces (RKHS) 4 Concentration inequalities 5 Subsampling of finite frames 6 L2 -Marcinkiewicz-Zygmund (MZ) inequalities 7 Least squares in the worst-case setting 8 Least squares in statistical learning 9 Cross-validation 10 Outlook