<p>  </p> <p>In the last years, lithium-ion batteries (LIBs) have become the most important energy storage system for consumer electronics, electric vehicles, and smart grids. The development of lithium-ion batteries (LIBs) based on current practice allows an energy density increase estimated at 10% per year. However, the required power for portable electronic devices is predicted to increase at a much faster rate, namely 20% per year. Similarly, the global electric vehicle battery capacity is expected to increase from around 170 GWh per year today to 1.5 TWh per year in 2030--this is an increase of 125% per year. Without a breakthrough in battery design technology, it will be difficult to keep up with the increasing energy demand. To that end, a design methodology to accelerate the LIB development is needed. This can be achieved through the integration of electro-chemical numerical simulations and machine learning algorithms.</p> <p><br></p> <p>To help this cause, this study develops a design methodology and framework using Simcenter Battery Design Studio® (BDS) and Bayesian optimization for design and optimization of cylindrical cell type 18650. The materials of the cathode are Nickel-Cobalt-Aluminum (NCA)/Nickel-Manganese-Cobalt-Aluminum (NMCA), anode is graphite, and electrolyte is Lithium hexafluorophosphate (LiPF6). Bayesian optimization has emerged as a powerful gradient-free optimization methodology to solve optimization problems that involve the evaluation of expensive black-box functions. The black-box functions are simulations of the cyclic performance test in Simcenter Battery Design Studio. </p> <p>The physics model used for this study is based on full system model described by Fuller and Newman. It uses Butler-Volmer Equation for ion-transportation across an interface and solvent diffusion model (Ploehn Model) for Aging of Lithium-Ion Battery Cells. The BDS model considers effects of SEI, cell electrode and microstructure dimensions, and charge-discharge rates to simulate battery degradation. Two objectives are optimized: maximization of the specific energy and minimization of the capacity fade. We perform global sensitivity analysis and see that thickness and porosity of the coating of the LIB electrodes that affect the objective functions the most. As such the design variables selected for this study are thickness and porosity of the electrodes. The thickness is restricted to vary from 22 micron to 240 microns and the porosity varies from 0.22 to 0.54. </p> <p>Two case studies are carried out using the above-mentioned objective functions and parameters. In the first study, cycling tests of 18650 NCA cathode Li-ion cells are simulated. The cells are charged and discharged using a constant 0.2C rate for 500 cycles. In the second case study a cathode active material more relevant to the electric vehicle industry, Nickel-Manganese-Cobalt-Aluminum (NMCA), is used. Here, the cells are cycled for 5 different charge-discharge scenarios to replicate charge-discharge scenario that an EVs battery module experiences. The results show that the design and optimization methodology can identify cells to satisfy the design objective that extend and improve the pareto front outside the original sampling plan for several practical charge-discharge scenarios which maximize energy density and minimize capacity fade. </p>

<p>The document describes the computational framework to optimize spring-driven Autoinjectors. It involves Bayesian Optimization for efficient and cost-effective design of Autoinjectors.</p>

Many algorithms today require a number of parameters to be set in order to perform well in a given application. The tuning of these parameters is often difficult and tedious to do manually, especially when the number of parameters is large. It is also unlikely that a human can find the best possible solution for difficult problems. To be able to automatically find good sets of parameters could both provide better results and save a lot of time. The prominent methods Bayesian optimization and Covariance Matrix Adaptation Evolution Strategy (CMA-ES) are evaluated for automatic parameter tuning in localization algorithms in this work. Both methods are evaluated using a localization algorithm on different datasets and compared in terms of computational time and the precision and recall of the final solutions. This study shows that it is feasible to automatically tune the parameters of localization algorithms using the evaluated methods. In all experiments performed in this work, Bayesian optimization was shown to make the biggest improvements early in the optimization but CMA-ES always passed it and proceeded to reach the best final solutions after some time. This study also shows that automatic parameter tuning is feasible even when using noisy real-world data collected from 3D cameras.

<div>Researchers and scientists across various areas face the perennial challenge of selecting experimental conditions or inputs for computer simulations in order to achieve promising results.</div><div> The aim of conducting these experiments could be to study the production of a material that has great applicability.</div><div> One might also be interested in accurately modeling and analyzing a simulation of a physical process through a high-fidelity computer code.</div><div> The presence of noise in the experimental observations or simulator outputs, called aleatory uncertainty, is usually accompanied by limited amount of data due to budget constraints.</div><div> This gives rise to what is known as epistemic uncertainty. </div><div> This problem of designing of experiments with limited number of allowable experiments or simulations under aleatory and epistemic uncertainty needs to be treated in a Bayesian way.</div><div> The aim of this thesis is to extend the state-of-the-art in Bayesian optimal design of experiments where one can optimize and infer statistics of the expensive experimental observation(s) or simulation output(s) under uncertainty.</div>

Ce travail de thèse porte sur le développement d’une méthodologie efficace pour la conception analogique, des algorithmes et des outils correspondants qui peuvent être utilisés dans la conception dynamique de fonctions linéaires à temps continu. L’objectif principal est d’assurer que les performances pour un système complet peuvent être rapidement investiguées, mais avec une précision comparable aux évaluations au niveau transistor.Une première direction de recherche a impliqué le développement de la méthodologie de conception basée sur le processus d'optimisation automatique de cellules au niveau transistor et la synthèse de macro-modèles analogiques de haut niveau dans certains environnements comme Mathworks - Simulink, VHDL-AMS ou Verilog-A. Le processus d'extraction des macro-modèles se base sur un ensemble complet d'analyses (DC, AC, transitoire, paramétrique, Balance Harmonique) qui sont effectuées sur les schémas analogiques conçues à partir d’une technologie spécifique. Ensuite, l'extraction et le calcul d'une multitude de facteurs de mérite assure que les modèles comprennent les caractéristiques de bas niveau et peuvent être directement régénéré au cours de l'optimisation.L'algorithme d'optimisation utilise une méthode bayésienne, où l'espace d’évaluation est créé à partir d'un modèle de substitution (krigeage dans ce cas), et la sélection est effectuée en utilisant le critère d’amélioration (Expected Improvement - EI) sujet à des contraintes. Un outil de conception a été développé (SIMECT), qui a été intégré comme une boîte à outils Matlab, employant les algorithmes d’extraction des macro-modèles et d'optimisation automatique. / The aim of this thesis is to establish an efficient analog design methodology, the algorithms and the corresponding design tools which can be employed in the dynamic conception of linear continuous-time (CT) functions. The purpose is to assure that the performance figures for a complete system can be rapidly investigated, but with comparable accuracy to the transistor-level evaluations. A first research direction implied the development of the novel design methodology based on the automatic optimization process of transistor-level cells using a modified Bayesian Kriging approach and the synthesis of robust high-level analog behavioral models in environments like Mathworks – Simulink, VHDL-AMS or Verilog-A.The macro-model extraction process involves a complete set of analyses (DC, AC, transient, parametric, Harmonic Balance) which are performed on the analog schematics implemented on a specific technology process. Then, the extraction and calculus of a multitude of figures of merit assures that the models include the low-level characteristics and can be directly regenerated during the optimization process.The optimization algorithm uses a Bayesian method, where the evaluation space is created by the means of a Kriging surrogate model, and the selection is effectuated by using the expected improvement (EI) criterion subject to constraints.A conception tool was developed (SIMECT), which was integrated as a Matlab toolbox, including all the macro-models extraction and automatic optimization techniques.

Optimization is an important part of mathematics and is mostly used for practical applications. For specific types of objective functions, a lot of different methods exist. A method to use when the objective is unknown and/or expensive can be difficult to determine. One of the answers is bayesian optimization, which instead of direct optimization creates a probabilistic model and uses it to constructs easily optimizable auxiliary function. It is an iterative method that uses information from previous iterations to find new point in which the objective is evaluated and tries to find the optimum within a fewer iterations. This thesis introduces bayesian optimization, suma- rizes its different approaches in lower and higher dimensions and shows when to use it suitably. An important part of the thesis is my own optimization algorithm which is applied to different practical problems - e.g. parameter optimization in machine learning algorithm. 1

Ce mémoire par article part de la question suivante: pouvons-nous utiliser des prothèses neurales afin d’activer artificiellement certain muscles dans le but d’accélérer la guérison et le réapprentissage du contrôle moteur après un AVC ou un traumatisme cervical ? Cette question touche plus de 15 millions de personnes chaque année à travers le monde, et est au coeur de la recherche de Numa Dancause et Marco Bonizzato, nos collaborateurs dans le département de Neuroscience de l’Université de Montréal. Il est maintenant possible d’implanter des électrodes à grande capacité dans le cortex dans le but d’acheminer des signaux électriques, mais encore difficile de prédire l’effet de stimulations sur le cerveau et le reste du corps. Cependant, des résultats préliminaires prometteurs sur des rats et singes démontrent qu’une récupération motrice non-négligeable est observée après stimulation de régions encore fonctionnelles du cortex moteur. Les difficultés rattachées à l’implémentation optimale de stimulation motocorticale consistent donc à trouver une de ces régions, ainsi qu’un protocole de stimulation efficace à la récupération. Bien que cette optimisation a été jusqu’à présent faite à la main, l’émergence d’implants capables de livrer des signaux sur plusieurs sites et avec plusieurs patrons spatio-temporels rendent l’exploration manuelle et exhaustive impossible. Une approche prometteuse afin d’automatiser et optimiser ce processus est d’utiliser un algorithme d’exploration bayésienne. Mon travail a été de déveloper et de raffiner ces techniques avec comme objectif de répondre aux deux questions scientifiques importantes suivantes: (1) comment évoquer des mouvements complexes en enchainant des microstimulations corticales ?, et (2) peuvent-elles avoir des effets plus significatifs que des stimulations simples sur la récupération motrice? Nous présentons dans l’article de ce mémoire notre approche hiérarchique utilisant des processus gaussiens pour exploiter les propriétés connues du cerveau afin d’accélérer la recherche, ainsi que nos premiers résultats répondant à la question 1. Nous laissons pour des travaux futur une réponse définitive à la deuxième question. / The idea for this thesis by article sprung from the following question: can we use neural prostheses to stimulate specific muscles in order to help recovery of motor control after stroke or cervical injury? This question is of crucial importance to 15 million people each year around the globe, and is at the heart of Numa Dancause and Marco Bonizzato’s research, our collaborators in the Neuroscience department at the University of Montreal. It is now possible to implant large capacity electrodes for electrical stimulation in cortex, but still difficult to predict their effect on the brain and the rest of the body. Nevertheless, preliminary but promising results on rats and monkeys have shown that a non-negligible motor recovery is obtained after stimulation of regions of motor cortex that are still functional. The difficulties related to optimal microcortical stimulation hence consist in finding both one of these regions, and a stimulation protocol with optimal recovery efficacy. This search has up to present day been performed by hand, but recent and upcoming large scale stimulation technologies permitting delivery of spatio-temporal signals are making such exhaustive searches impossible.A promising approach to automating and optimizing this discovery is the use of Bayesian optimization. My work has consisted in developing and refining such techniques with two scientific questions in mind: (1) how can we evoke complex movements by chaining cortical microstimulations?, and (2) can these outperform single channel stimulations in terms of recovery efficacy? We present in the main article of this thesis our hierarchical Bayesian optimization approach which uses gaussian processes to exploit known properties of the brain to speed up the search, as well as first results answering question 1. We leave to future work a definitive answer to the second question.

Machine learning for image classification is a hot topic and it is increasing in popularity. Therefore the aim of this study is to provide a better understanding of convolutional neural network hyperparameters by comparing the test accuracy of convolutional neural network models with different hyperparameter value configurations. The focus of this study is to see whether there is an influence in the learning process depending on which hyperparameter values were used. For conducting the experiments convolutional neural network models were developed using the programming language Python utilizing the library Keras. The dataset used for this study iscifar-10, it includes 60000 colour images of 10 categories ranging from man-made objects to different animal species. Grid search is used for instantiating models with varying learning rate and momentum, width and depth values. Learning rate is only tested combined with momentum and width is only tested combined with depth. Activation functions, convolutional layers and batch size are tested individually. Grid search is compared against Bayesian optimization to see which technique will find the most optimized learning rate and momentum values. Results illustrate that the impact different hyperparameters have on the overall test accuracy varies. Learning rate and momentum affects the test accuracy greatly, however suboptimal values for learning rate and momentum can decrease the test accuracy severely. Activation function, width and depth, convolutional layer and batch size have a lesser impact on test accuracy. Regarding Bayesian optimization compared to grid search, results show that Bayesian optimization will not necessarily find more optimal hyperparameter values.

<p>Practical design problems in engineering and science involve the evaluation of expensive black-box functions, the optimization of multiple—often conflicting—targets, and the integration of data generated by multiple sources of information, e.g., numerical models with different levels of fidelity. If not properly handled, the complexity of these design problems can lead to lengthy and costly development cycles. In the last years, Bayesian optimization has emerged as a powerful alternative to solve optimization problems that involve the evaluation of expensive black-box functions. Bayesian optimization has two main components: a probabilistic surrogate model of the black-box function and an acquisition function that drives the optimization. Its ability to find high-performance designs within a limited number of function evaluations has attracted the attention of many fields including the engineering design community. The practical relevance of strategies with the ability to fuse information emerging from different sources and the need to optimize multiple targets has motivated the development of multi-fidelity modeling techniques and multi-objective Bayesian optimization methods. A key component in the vast majority of these methods is the Gaussian process (GP) due to its flexibility and mathematical properties.</p> <p><br></p> <p>The objective of this dissertation is to develop new approaches in the areas of multi-fidelity modeling and multi-objective Bayesian optimization. To achieve this goal, this study explores the use of linear and non-linear compositions of GPs to build probabilistic models for Bayesian optimization. Additionally, motivated by the rationale behind well-established multi-objective methods, this study presents a novel acquisition function to solve multi-objective optimization problems in a Bayesian framework. This dissertation presents four contributions. First, the auto-regressive model, one of the most prominent multi-fidelity models in engineering design, is extended to include informative mean functions that capture prior knowledge about the global trend of the sources. This additional information enhances the predictive capabilities of the surrogate. Second, the non-linear auto-regressive Gaussian process (NARGP) model, a non-linear multi-fidelity model, is integrated into a multi-objective Bayesian optimization framework. The NARGP model offers the possibility to leverage sources that present non-linear cross-correlations to enhance the performance of the optimization process. Third, GP classifiers, which employ non-linear compositions of GPs, and conditional probabilities are combined to solve multi-objective problems. Finally, a new multi-objective acquisition function is presented. This function employs two terms: a distance-based metric—the expected Pareto distance change—that captures the optimality of a given design, and a diversity index that prevents the evaluation of non-informative designs. The proposed acquisition function generates informative landscapes that produce Pareto front approximations that are both broad and diverse.</p>

[pt] Os risers são importantes componentes na produção e exploração de petróleo e derivados. São responsáveis pelo transporte do óleo e gás encontrados no reservatório até a Unidade Estacionária de Produção (UEP) ou pela injeção de gás ou água no reservatório. A crescente demanda por esse produto faz com que a exploração seja feita em regiões com condições cada vez mais adversas. Tipicamente, um projeto deste porte exige um número muito grande de análises numéricas de elementos finitos e exigem uma experiência grande por parte do projetista a fim de obter uma solução viável. Esse desafio leva engenheiros a buscarem ferramentas consistentes e seguras que auxiliem nas etapas iniciais do projeto das configurações de risers e que sejam capazes de diminuir o número de análises totais exigidas. Uma dessas ferramentas é a utilização de métodos de otimização para obter de maneira consistente e segura os parâmetros que definem uma configuração. Este trabalho apresenta o método de Otimização Bayesiana, um método baseado em técnicas de aprendizado de máquina capaz de resolver problemas de otimização do tipo caixa-preta de maneira eficiente explorando o uso de aproximações analíticas da função objetivo, que se quer otimizar. O método é aplicado em diferentes estudos de casos visando validálo como capaz de resolver problemas de configuração de riser de maneira eficiente e consistente. Dentre os problemas aplicados estão diferentes tipos de configurações, diferentes casos realistas, mono-objetivo e multi-objetivo. / [en] Risers are an important component in the oil s production and exploration field. They are responsible for the oil and gas transportation from the reservoir to the floating unit or injection of gas or water into the reservoir. The increasing the demand for this product has lead projects to explore to areas in which conditions are harsher. Typically, such a large project demands a large number of numerical finite element analyses and a great expertise from the engineer in charge in order to obtain a viable solution. This challenge leads engineers in search of consistent and reliable tools that assist in the early stages of the riser configuration design and are capable of reducing the number of total analyses required. One of these tools is application of optimization methods to obtain in a consistent and reliable manner the parameters which define a configuration. This work presents the Bayesian Optimization method, a method based on machine learning techniques capable of efficiently solving so called black box problems by exploring analytical approximations of the objective function, the function to be minimized. The method is applied to different case studies aiming to validate it as capable of solving a wide variety of riser configuration problems in an efficient and consistent way. Among the problems applied are different types of configurations, different realistic cases, mono-objective and multi-objective.