Spatio-temporal data is a commonly used source of information. Using machine learning to analyse this kind of data can lead to many interesting and useful insights. In this thesis project, a novel public transportation spatio-temporal dataset is explored and analysed. The dataset contains 282 GB of positional events, spanning two weeks of time, from all public transportation vehicles in Östergötland county, Sweden.  From the data exploration, three high-level problems are formulated: bus stop detection, GPS variance estimation, and arrival time prediction, also called trajectory forecasting. The bus stop detection problem is briefly discussed and solutions are proposed. Gaussian process regression is an effective method for solving regression problems. The GPS variance estimation problem is solved via the use of a mixture of Gaussian processes. A mixture of Gaussian processes is also used to predict the arrival time for public transportation buses. The arrival time prediction is from one bus stop to the next, not for the whole trajectory.  The result from the arrival time prediction is a distribution of arrival times, which can easily be applied to determine the earliest and latest expected arrival to the next bus stop, alongside the most probable arrival time. The naïve arrival time prediction model implemented has a root mean square error of 5 to 19 seconds. In general, the absolute error of the prediction model decreases over time in each respective segment. The results from the GPS variance estimation problem is a model which can compare the variance for different environments along the route on a given trajectory.

L'oculométrie est un ensemble de techniques dédié à enregistrer et analyser les mouvements oculaires. Dans cette thèse, je présente l'étude, la conception et la mise en œuvre d'un système oculométrique numérique, non-intrusif permettant d'analyser les mouvements oculaires en temps réel avec une webcam à distance et sans lumière infra-rouge. Dans le cadre de la réalisation, le système oculométrique proposé se compose de quatre modules: l'extraction des caractéristiques, la détection et le suivi des yeux, l'analyse de la variété des mouvements des yeux à partir des images et l'estimation du regard par l'apprentissage. Nos contributions reposent sur le développement des méthodes autour de ces quatre modules: la première réalise une méthode hybride pour détecter et suivre les yeux en temps réel à partir des techniques du filtre particulaire, du modèle à formes actives et des cartes des yeux (EyeMap); la seconde réalise l'extraction des caractéristiques à partir de l'image des yeux en utilisant les techniques des motifs binaires locaux; la troisième méthode classifie les mouvements oculaires selon la variété générée par le Laplacian Eigenmaps et forme un ensemble de données d'apprentissage; enfin, la quatrième méthode calcul la position du regard à partir de cet ensemble d'apprentissage. Nous proposons également deux méthodes d'estimation:une méthode de la régression par le processus gaussien et un apprentissage semi-supervisé et une méthode de la catégorisation par la classification spectrale (spectral clustering). Il en résulte un système complet, générique et économique pour les applications diverses dans le domaine de l'oculométrie. / Gaze tracker offers a powerful tool for diverse study fields, in particular eye movement analysis. In this thesis, we present a new appearance-based real-time gaze tracking system with only a remote webcam and without infra-red illumination. Our proposed gaze tracking model has four components: eye localization, eye feature extraction, eye manifold learning and gaze estimation. Our research focuses on the development of methods on each component of the system. Firstly, we propose a hybrid method to localize in real time the eye region in the frames captured by the webcam. The eye can be detected by Active Shape Model and EyeMap in the first frame where eye occurs. Then the eye can be tracked through a stochastic method, particle filter. Secondly, we employ the Center-Symmetric Local Binary Patterns for the detected eye region, which has been divided into blocs, in order to get the eye features. Thirdly, we introduce manifold learning technique, such as Laplacian Eigen-maps, to learn different eye movements by a set of eye images collected. This unsupervised learning helps to construct an automatic and correct calibration phase. In the end, as for the gaze estimation, we propose two models: a semi-supervised Gaussian Process Regression prediction model to estimate the coordinates of eye direction; and a prediction model by spectral clustering to classify different eye movements. Our system with 5-points calibration can not only reduce the run-time cost, but also estimate the gaze accurately. Our experimental results show that our gaze tracking model has less constraints from the hardware settings and it can be applied efficiently in different real-time applications.

Cette thèse présente un framework d’optimisation pour la conception hydrodynamique de forme d’hydrofoils. L’optimisation d’hydrofoil par simulation implique des objectifs d’optimisation divergents et impose des compromis contraignants en raison du coût des simulations numériques et des budgets limités généralement alloués à la conception des navires. Le framework fait appel à l’échantillonnage séquentiel et aux modèles de substitution. Un modèle prédictif est construit en utilisant la Régression par Processus Gaussien (RPG) à partir des données issues de simulations fluides effectuées sur différentes géométries d’hydrofoils. Le modèle est ensuite combiné à d’autres critères dans une fonction d’acquisition qui est évaluée sur l’espace de conception afin de définir une nouvelle géométrie qui est testée et dont les paramètres et la réponse sont ajoutés au jeu de données, améliorant ainsi le modèle. Une nouvelle fonction d’acquisition a été développée, basée sur la variance RPG et la validation croisée des données. Un modeleur géométrique a également été développé afin de créer automatiquement les géométries d’hydrofoil a partir des paramètres déterminés par l’optimiseur. Pour compléter la boucle d’optimisation,FINE/Marine, un solveur fluide RANS, a été intégré dans le framework pour exécuter les simulations fluides. Les capacités d’optimisation ont été testées sur des cas tests analytiques montrant que la nouvelle fonction d’acquisition offre plus de robustesse que d’autres fonctions d’acquisition existantes. L’ensemble du framework a ensuite été testé sur des optimisations de sections 2Dd’hydrofoil ainsi que d’hydrofoil 3D avec surface libre. Dans les deux cas, le processus d’optimisation fonctionne, permettant d’optimiser les géométries d’hydrofoils et confirmant les performances obtenues sur les cas test analytiques. Les optima semblent cependant être assez sensibles aux conditions opérationnelles. / This thesis presents a practical hydrodynamic optimization framework for hydrofoil shape design. Automated simulation based optimization of hydrofoil is a challenging process. It may involve conflicting optimization objectives, but also impose a trade-off between the cost of numerical simulations and the limited budgets available for ship design. The optimization frameworkis based on sequential sampling and surrogate modeling. Gaussian Process Regression (GPR) is used to build a predictive model based on data issued from fluid simulations of selected hydrofoil geometries. The GPR model is then combined with other criteria into an acquisition function that isevaluated over the design space, to define new querypoints that are added to the data set in order to improve the model. A custom acquisition function is developed, based on GPR variance and cross validation of the data.A hydrofoil geometric modeler is also developed to automatically create the hydrofoil shapes based on the parameters determined by the optimizer. To complete the optimization loop, FINE/Marine, a RANS flow solver, is embedded into the framework to perform the fluid simulations. Optimization capabilities are tested on analytical test cases. The results show that the custom function is more robust than other existing acquisition functions when tested on difficult functions. The entire optimization framework is then tested on 2D hydrofoil sections and 3D hydrofoil optimization cases with free surface. In both cases, the optimization process performs well, resulting in optimized hydrofoil shapes and confirming the results obtained from the analytical test cases. However, the optimum is shown to be sensitive to operating conditions.

Gaussian Processes (GPs) are promising Bayesian methods for classification and regression problems. They have also been used for semi-supervised classification tasks. In this thesis, we propose new algorithms for solving semi-supervised binary classification problem using GP regression (GPR) models. The algorithms are closely related to semi-supervised classification based on support vector regression (SVR) and maximum margin clustering. The proposed algorithms are simple and easy to implement. Also, the hyper-parameters are estimated without resorting to expensive cross-validation technique. The algorithm based on sparse GPR model gives a sparse solution directly unlike the SVR based algorithm. Use of sparse GPR model helps in making the proposed algorithm scalable. The results of experiments on synthetic and real-world datasets demonstrate the efficacy of proposed sparse GP based algorithm for semi-supervised classification.

Optimization is an important part of mathematics and is mostly used for practical applications. For specific types of objective functions, a lot of different methods exist. A method to use when the objective is unknown and/or expensive can be difficult to determine. One of the answers is bayesian optimization, which instead of direct optimization creates a probabilistic model and uses it to constructs easily optimizable auxiliary function. It is an iterative method that uses information from previous iterations to find new point in which the objective is evaluated and tries to find the optimum within a fewer iterations. This thesis introduces bayesian optimization, suma- rizes its different approaches in lower and higher dimensions and shows when to use it suitably. An important part of the thesis is my own optimization algorithm which is applied to different practical problems - e.g. parameter optimization in machine learning algorithm. 1

With advancements in technology, the dimensions of transistors are scaling down. It leads to shrinkage in the size of memory bitcells, increasing its sensitivity to process variations introduced during manufacturing. Failure of a single bitcell can cause the failure of an entire memory; hence careful statistical analysis is essential in estimating the highest reliable performance of the bitcell before using them in memory design. With high repetitiveness of bitcell, the traditional method of Monte Carlo simulation would require along time for accurate estimation of rare failure events. A more practical approach is importance sampling where more samples are collected from the failure region. Even though importance sampling is much faster than Monte Carlo simulations, it is still fairly time-consuming as it demands an iterative search making it impractical for large simulation sets. This thesis proposes two machine learning models that can be used in estimating the performance of a bitcell. The first model predicts the time taken by the bitcell for read or write operation. The second model predicts the minimum voltage required in maintaining the bitcell stability. The models were trained using the K-nearest neighbors algorithm and Gaussian process regression. Three sparse approximations were implemented in the time prediction model as a bigger dataset was available. The obtained results show that the models trained using Gaussian process regression were able to provide promising results.

A unifying model for Critical Heat Flux (CHF) prediction has been elusive for over 60 years. With the release of the data utilized in the making of the 2006 Groeneveld Lookup table (LUT), by far the largest public CHF database available to date, data-driven predictions on a large variable space can be performed. The popularization of machine learning techniques to solve regression problems allows for deeper and more advanced tools when analyzing the data. We compare three different machine learning algorithms to predict the occurrence of CHF in vertical, uniformly heated round tubes. For each selected algorithm (ν-Support vector regression, Gaussian process regression, and Neural network regression), an optimized hyperparameter set is fitted. The best performing algorithm is the Neural network, which achieves a standard deviation of the prediction/measured factor three times lower than the LUT, while the Gaussian process regression and the ν-Support vector regression both lead to two times lower standard deviation. All algorithms significantly outperform the LUT prediction performance. The neural network model and training methodology are designed to prevent overfitting, which is confirmed by data analysis of the predictions. Additionally, a feasibility study of transfer learning and uncertainty quantification is performed, to investigate potential future applications.

<p>Lithium-Ion Batteries (Li-ion batteries or LIBs) have been extensively used in a wide variety of industrial applications and consumer electronics. Additive Manufacturing (AM) or 3D printing (3DP) techniques have evolved to allow the fabrication of complex structures of various compositions in a wide range of applications. </p> <p><br></p> <p>The objective of the thesis is to investigate the application of 3DP to fabricate a LIB, using a modified process from the literature [1]. The ultimate goal is to improve the electrochemical performances of LIBs while maintaining design flexibility with a 3D printed 3D architecture. </p> <p><br></p> <p>In this research, both the cathode and anode in the form of specifically formulated slurry were extruded into filaments using a high-temperature pellet-based extruder. Specifically, filament composites made of graphite and Polylactic Acid (PLA) were fabricated and tested to produce anodes. Investigations on two other types of PLA-based filament composites respectively made of Lithium Manganese Oxide (LMO) and Lithium Nickel Manganese Cobalt Oxide (NMC) were also conducted to produce cathodes. Several filaments with various materials ratios were formulated in order to optimize printability and battery capacities. Finally, flat battery electrode disks similar to conventional electrodes were fabricated using the fused deposition modeling (FDM) process and assembled in half-cells and full cells. Finally, the electrochemical properties of half cells and full cells were characterized. Additionally, in parallel to the experiment, a 1-D finite element (FE) model was developed to understand the electrochemical performance of the anode half-cells made of graphite. Moreover, a simplified machine learning (ML) model through the Gaussian Process Regression was used to predict the voltage of a certain half-cell based on input parameters such as charge and discharge capacity. </p> <p><br></p> <p>The results of this research showed that 3D printing technology is capable to fabricate LIBs. For the 3D printed LIB, cells have improved electrochemical properties by increasing the material content of active materials (i.e., graphite, LMO, and NMC) within the PLA matrix, along with incorporating a plasticizer material. The FE model of graphite anode showed a similar trend of discharge curve as the experiment. Finally, the ML model demonstrated a reasonably good prediction of charge and discharge voltages. </p>

This document is a Master Thesis report in financial mathematics for KTH. This Master thesis is the product of an internship conducted at Nexialog Consulting, in Paris. This document is about the innovative use of Constrained Gaussian process regression in order to build an arbitrage free swaption cube. The methodology introduced in the document is used on a data set of European Swaptions Out of the Money. / Det här dokumentet är en magisteruppsats i finansiel matematik på KTH. Detta examensarbete är resultatet av en praktik som ufördes på Nexialog Consulting i Paris.Detta dokument handlar om den innovativa användningen av regression för gaussiska processer med bivillkor för att bygga en arbitragefri swaption kub. Den metodik som introduceras i dokumentet används på en datamängd av europeiska swaptions som är "Out of the Money".

Being able to keep track of ones assets is a very useful thing, from avoiding losing ones keys or phone to being able to find the needed equipment in a busy hospital or on a construction site. The area of localization is actively evolving to find the best ways to accurately track objects and devices in an energy efficient manner, at any range, and in any type of environment. This thesis focuses on the last aspect of maintaining accurate localization regardless of environment. For radio frequency based systems, challenging environments containing many obstacles, e.g., indoor or urban areas, have a detrimental effect on the measurements used for positioning, making them deceptive. In this work, a method for correcting range measurements is proposed for a narrowband sub-GHz radio frequency based localization system using Received Signal Strength Indicator (RSSI) and Time-of-Flight (ToF) measurements for positioning. Three different machine learning models were implemented: a linear regressor, a least squares support vector machine regressor and a gaussian process regressor. They were compared in their ability to predict the true range between devices based on raw range measurements. Achieved was a 69.96 % increase in accuracy compared to uncorrected ToF estimates and a 88.74 % increase in accuracy compared to RSSI estimates. When the corrected range estimates were used for positioning with a trilateration algorithm using least squares estimation, a 67.84 % increase in accuracy was attained compared to positioning with uncorrected range estimates. This shows that this is an effective method of improving range estimates to facilitate more accurate positioning. / Att kunna hålla reda på var ens tillgångar befinner sig kan vara mycket användbart, från att undvika att ens nycklar eller telefon tappas bort till att kunna hitta utrustningen man behöver i ett myllrande sjukhus eller på en byggarbetsplats. Området av lokalisering utvecklas aktivt för att hitta de bästa metoderna och teknologierna för att med precision kunna spåra fysiska objekt på ett energieffektivt sätt, på vilken räckvidd som helst, och i vilken miljö som helst. Detta arbete fokuserar på den sista aspekten av att uppnå precis positionering oavsett miljö. För radiofrekvensbaserade system har utmanande miljöer med många fysiska hinder som till exempel inomhus och stadsområden en negativ effekt på de mätningar som används för positionering, vilket gör dem vilseledande. I detta arbete föreslås en metod för att korrigera avståndsmätningar i ett narrowband sub-GHz radiofrekvensbaserat lokaliseringssystem som använder Received Signal Strength Indicator (RSSI)- och Time-of-Flight (ToF)-mätningar för positionering. Tre olika maskininlärningsmodeller har implementerats: en linear regressor, en least squares support vector machine regressor och en gaussian process regressor. Dessa jämfördes i sin förmåga att förutspå det sanna avståndet mellan enheter baserat på råa avståndsmätningar. De korrigerade avståndsmätningarna uppnådde 69.96 % högre nogrannhet jämfört med okorrigerade ToF-uppskattningar och 88.74 % högre nogrannhet jämfört med RSSI-uppskattningar. Avståndsuppskattningarna användes för positionering med trilateration och minsta kvadratmetoden. De korrigerade uppskattningarna gav 67.84 % mer precis positionering jämfört med de okorrigerde uppskattningarna. Detta visar att detta är en effektiv metod förbättra avståndsuppskattningarna för att i sin tur bidra till mer exakt positionering.