Evolutionary algorithms have proved to be useful for tackling many practical black-box optimization problems. In this thesis, we describe one of the most powerful evolutionary algorithms of today, CMA- ES, and apply it in novel way to solve the problem of tuning multiple coupled PID controllers in combustion engine models. Powered by TCPDF (www.tcpdf.org)

This thesis introduces a new optimization algorithm for simulation-based design of systems with multi-modal, nonlinear, black box objective functions. The algorithm extends the recently introduced adaptive multi-modal optimization by incorporating surrogate modeling features similar to response surface methods (RSM). The resulting optimization algorithm has reduced computational intensity and is therefore well-suited for optimization of expensive black box objective functions. The algorithm relies on an adaptive and multi-resolution mesh to obtain an initial estimation of the objective function surface. Local surrogate models are then constructed to represent the objective function and to generate additional trial points in the vicinity of local minima discovered. The steps of mesh refinement and surrogate modeling continue until convergence criteria are met. An important property of this algorithm is that it produces progressively accurate surrogate models around the local minima; these models can be used for post-optimization studies such as sensitivity and tolerance analyses with minimal computational effort. This algorithm is suitable for optimal design of complex engineering systems and enhances the design cycle by enabling computationally affordable uncertainty analysis. The mathematical basis of the algorithm is explained in detail. The thesis also demonstrates the effectiveness of the algorithm using comparative optimization of several multi-modal objective functions. It also shows several practical applications of the algorithm in the design of complex power and power-electronic systems.

This thesis introduces a new optimization algorithm for simulation-based design of systems with multi-modal, nonlinear, black box objective functions. The algorithm extends the recently introduced adaptive multi-modal optimization by incorporating surrogate modeling features similar to response surface methods (RSM). The resulting optimization algorithm has reduced computational intensity and is therefore well-suited for optimization of expensive black box objective functions. The algorithm relies on an adaptive and multi-resolution mesh to obtain an initial estimation of the objective function surface. Local surrogate models are then constructed to represent the objective function and to generate additional trial points in the vicinity of local minima discovered. The steps of mesh refinement and surrogate modeling continue until convergence criteria are met. An important property of this algorithm is that it produces progressively accurate surrogate models around the local minima; these models can be used for post-optimization studies such as sensitivity and tolerance analyses with minimal computational effort. This algorithm is suitable for optimal design of complex engineering systems and enhances the design cycle by enabling computationally affordable uncertainty analysis. The mathematical basis of the algorithm is explained in detail. The thesis also demonstrates the effectiveness of the algorithm using comparative optimization of several multi-modal objective functions. It also shows several practical applications of the algorithm in the design of complex power and power-electronic systems.

In this thesis, the theory and application of black-box optimization methods are explored. More specifically, we looked at two families of algorithms, descent methods andresponse surface methods (closely related to trust region methods). We also looked at possibilities in using a dimension reduction technique called active subspace which utilizes sampled gradients. This dimension reduction technique can make the descent methods more suitable to high-dimensional problems, which turned out to be most effective when the data have a ridge-like structure. Finally, the optimization methods were used on a real-world problem in the context of pre-hospital care where the objective is to minimize the ambulance response times in the municipality of Umea by changing the positions of the ambulances. Before applying the methods on the real-world ambulance problem, a simulation study was performed on synthetic data, aiming at finding the strengths and weaknesses of the different models when applied to different test functions, at different levels of noise. The results showed that we could improve the ambulance response times across several different performance metrics compared to the response times of the current ambulancepositions. This indicates that there exist adjustments that can benefit the pre-hospitalcare in the municipality of Umea. However, since the models in this thesis work find local and not global optimums, there might still exist even better ambulance positions that can improve the response time further. / I denna rapport undersöks teorin och tillämpningarna av diverse blackbox optimeringsmetoder. Mer specifikt så har vi tittat på två familjer av algoritmer, descentmetoder och responsytmetoder (nära besläktade med tillitsregionmetoder). Vi tittar också på möjligheterna att använda en dimensionreduktionsteknik som kallas active subspace som använder samplade gradienter för att göra descentmetoderna mer lämpade för högdimensionella problem, vilket visade sig vara mest effektivt när datat har en struktur där ändringar i endast en riktning har effekt på responsvärdet. Slutligen användes optimeringsmetoderna på ett verkligt problem från sjukhusvården, där målet var att minimera svarstiderna för ambulansutryckningar i Umeå kommun genom att ändra ambulanspositionerna. Innan metoderna tillämpades på det verkliga ambulansproblemet genomfördes också en simuleringsstudie på syntetiskt data. Detta för att hitta styrkorna och svagheterna hos de olika modellerna genom att undersöka hur dem hanterar ett flertal testfunktioner under olika nivåer av brus. Resultaten visade att vi kunde förbättra ambulansernas responstider över flera olika prestandamått jämfört med responstiderna för de nuvarande ambulanspositionerna. Detta indikerar att det finns förändringar av positioneringen av ambulanser som kan gynna den pre-hospitala vården inom Umeå kommun. Dock, eftersom modellerna i denna rapport hittar lokala och inte globala optimala punkter kan det fortfarande finnas ännu bättre ambulanspositioner som kan förbättra responstiden ytterligare.

The simulation program GTperform is used to estimate the machine settings from performance measurements for the gas turbine model STG-800 at Siemens Industrial Turbomachinery in Finspång, Sweden. By evaluating different settings within the program, the engineers try to estimate the one that generatesthe performance measurement. This procedure is done manually at Siemens and is very time-consuming. This project aims to establish an algorithm that automatically establishes the correct machine setting from the performance measurements. Two algorithms were implemented in Python: Simulated Annealing and Gradient Descent. The algorithms analyzed two possible objective functions, and objective were tested on three gas turbines located at different locations. The first estimated the machine setting that generated the best fit to the performance measurements, while the second established the most likely solution for the machine setting from probability distributions. Multiple simulations have been run for the two algorithms and objective functions to evaluate the performances. Both algorithms successfully established satisfactory results for the second objective function. The Simulated Annealing, in particular, established solutions with a lower spread compared to Gradient Descent. The algorithms give a possibility to automatically establish the machine settings for the simulation program, reducing the work for the engineers.

Designing products and processes is a fundamental aspect of engineering that significantly impacts society and the world. Chemical process design aims to create more efficient and sustainable production processes that consume fewer resources and emit less pollution. Mathematical models that accurately describe process behavior are necessary to make informed and responsible decisions. However, as processes become more complex, purely symbolic formulations may be inadequate, and simulations using tailored computer code become necessary. The decision‐making process in optimal design requires a procedure for choosing the best option while complying with the system’s constraints, for which task optimization approaches are well suited. This doctoral thesis focuses on black‐box optimization problems that arise when using process simulators in optimal process design tasks and assesses the potential of derivative‐free, metaheuristics, and surrogate‐based optimization approaches. The optimal design of natural gas liquefaction processes is the case study of this research. To overcome numerical issues from black‐box problems, the first work of this doctoral thesis consisted of using the globally convergent Nelder‐Mead simplex method to the optimal process design problem. The second work introduced surrogate models to assist the search towards the global optimum of the black‐box problem and an adaptive sampling scheme comprising the optimization of an acquisition function with metaheuristics. Kriging as surrogate models to the simulation‐optimization problems are computationally cheaper and effective predictors suitable for global search. The third work aims to overcome the limitations of acquisition function optimization and the use of metaheuristics. The proposed comprehensive mathematical notation of the surrogate optimization problem was readily implementable in algebraic modeling language software. The presented framework includes kriging models of the objective and constraint functions, an adaptive sampling procedure, a heuristic for stopping criteria, and a readily solvable surrogate optimization problem with mathematical programming. The success of the surrogate‐based optimization framework relies on the kriging models’ prediction accuracy regarding the underlying, simulation‐based functions. The fourth publication extends the previous work to multi‐objective black‐box optimization problems. It applies the ε constraint method to transform the multi‐objective surrogate optimization problem into a sequence of single‐objective ones. The ε‐constrained surrogate optimization problems are implemented automatically in algebraic modeling language software and solved using a gradient‐based, state‐of‐the‐art solver. The fifth publication is application-driven and focuses on identifying the most suitable mixed‐refrigerant refrigeration technology for natural gas liquefaction in terms of energy consumption and costs. The study investigates five natural gas liquefaction processes using particle swarm optimization and concludes that there are flaws in the expected relationships between process complexity, energy consumption, and total annualized costs. In conclusion, the research conducted in this doctoral thesis demonstrates the importance and capabilities of using optimization to process simulators. The work presented here highlights the potential of surrogate‐based optimization approaches to significantly reduce the computational cost and guide the search in black‐box optimization problems with chemical process simulators embedded. Overall, this doctoral thesis contributes to developing optimization strategies for complex chemical processes that are essential for addressing some of the current most pressing environmental and social challenges. The methods and insights presented in this work can help engineers and scientists design more sustainable and efficient processes, contributing to a better future for all.

Microsized hexagonal gallium nitride pyramids show promise as next generation Light Emitting Diodes (LEDs) due to certain quantum properties within the pyramids. One metric for evaluating the efficiency of a LED device is by studying its Light Extraction Efficiency (LEE). To calculate the LEE for different pyramid designs, simulations can be performed using the FDTD method. Maximizing the LEE is treated as a black-box optimization problem with an interpolation method that utilizes radial basis functions. A simple heuristic is implemented and tested for various pyramid parameters. The LEE is shown to be highly dependent on the pyramid size, the source position and the polarization. Under certain circumstances, a LEE over 17% is found above the pyramid. The results are however in some situations very sensitive to the simulation parameters, leading to results not converging properly. Establishing convergence for all simulation evaluations must be done with further care. The results imply a high LEE for the pyramids is possible, which motivates the need for further research.

Many algorithms today require a number of parameters to be set in order to perform well in a given application. The tuning of these parameters is often difficult and tedious to do manually, especially when the number of parameters is large. It is also unlikely that a human can find the best possible solution for difficult problems. To be able to automatically find good sets of parameters could both provide better results and save a lot of time. The prominent methods Bayesian optimization and Covariance Matrix Adaptation Evolution Strategy (CMA-ES) are evaluated for automatic parameter tuning in localization algorithms in this work. Both methods are evaluated using a localization algorithm on different datasets and compared in terms of computational time and the precision and recall of the final solutions. This study shows that it is feasible to automatically tune the parameters of localization algorithms using the evaluated methods. In all experiments performed in this work, Bayesian optimization was shown to make the biggest improvements early in the optimization but CMA-ES always passed it and proceeded to reach the best final solutions after some time. This study also shows that automatic parameter tuning is feasible even when using noisy real-world data collected from 3D cameras.

Názov práce: Evoluční algoritmy a aktivní učení Autor: Jakub Repický Katedra: Katedra teoretické informatiky a matematické logiky Vedúci diplomovej práce: doc. RNDr. Ing. Martin Holeňa, CSc., Ústav informa- tiky, Akademie věd České republiky Abstrakt: Vyhodnotenie ciel'ovej funkcie v úlohách spojitej optimalizácie často do- minuje výpočtovej náročnosti algoritmu. Platí to najmä v prípade black-box fun- kcií, t. j. funkcií, ktorých analytický popis nie je známy a ktoré sú vyhodnocované empiricky. Témou urýchl'ovania black-box optimalizácie s pomocou náhradných modelov ciel'ovej funkcie sa zaoberá vel'a autorov a autoriek. Ciel'om tejto dip- lomovej práce je vyhodnotit' niekol'ko metód, ktoré prepájajú náhradné modely založené na Gaussovských procesoch (GP) s Evolučnou stratégiou adaptácie ko- variančnej matice (CMA-ES). Gaussovské procesy umožňujú aktívne učenie, pri ktorom sú body pre vyhodnotenie vyberané s ciel'om zlepšit' presnost' modelu. Tradičné náhradné modely založené na GP zah'rňajú Metamodelom asistovanú evolučnú stratégiu (MA-ES) a Optimalizačnú procedúru pomocou Gaussovských procesov (GPOP). Pre účely tejto práce boli oba prístupy znovu implementované a po prvý krát vyhodnotené na frameworku Black-Box...

On considère ici deux sujets différents, en utilisant des idées issues de la théorie de l'information : 1) Context Tree Weighting est un algorithme de compression de texte qui calcule exactement une prédiction Bayésienne qui considère tous les modèles markoviens visibles : on construit un "arbre de contextes", dont les nœuds profonds correspondent aux modèles complexes, et la prédiction est calculée récursivement à partir des feuilles. On étend cette idée à un contexte plus général qui comprend également l'estimation de densité et la régression, puis on montre qu'il est intéressant de remplacer les mixtures Bayésiennes par du "switch", ce qui revient à considérer a priori des suites de modèles plutôt que de simples modèles. 2) Information Geometric Optimization (IGO) est un cadre général permettant de décrire plusieurs algorithmes d'optimisation boîte noire, par exemple CMA-ES et xNES. On transforme le problème initial en un problème d'optimisation d'une fonction lisse sur une variété Riemannienne, ce qui permet d'obtenir une équation différentielle du premier ordre invariante par reparamétrage. En pratique, il faut discrétiser cette équation, et l'invariance n'est plus valable qu'au premier ordre. On définit l'algorithme IGO géodésique (GIGO), qui utilise la structure de variété Riemannienne mentionnée ci-dessus pour obtenir un algorithme totalement invariant par reparamétrage. Grâce au théorème de Noether, on obtient facilement une équation différentielle du premier ordre satisfaite par les géodésiques de la variété statistique des gaussiennes, ce qui permet d'implémenter GIGO. On montre enfin que xNES et GIGO sont différents dans le cas général, mais qu'il est possible de définir un nouvel algorithme presque invariant par reparamétrage, GIGO par blocs, qui correspond exactement à xNES dans le cas Gaussien. / We study two different topics, using insight from information theory in both cases: 1) Context Tree Weighting is a text compression algorithm that efficiently computes the Bayesian combination of all visible Markov models: we build a "context tree", with deeper nodes corresponding to more complex models, and the mixture is computed recursively, starting with the leaves. We extend this idea to a more general context, also encompassing density estimation and regression; and we investigate the benefits of replacing regular Bayesian inference with switch distributions, which put a prior on sequences of models instead of models. 2) Information Geometric Optimization (IGO) is a general framework for black box optimization that recovers several state of the art algorithms, such as CMA-ES and xNES. The initial problem is transferred to a Riemannian manifold, yielding parametrization-invariant first order differential equation. However, since in practice, time is discretized, this invariance only holds up to first order. We introduce the Geodesic IGO (GIGO) update, which uses this Riemannian manifold structure to define a fully parametrization invariant algorithm. Thanks to Noether's theorem, we obtain a first order differential equation satisfied by the geodesics of the statistical manifold of Gaussians, thus allowing to compute the corresponding GIGO update. Finally, we show that while GIGO and xNES are different in general, it is possible to define a new "almost parametrization-invariant" algorithm, Blockwise GIGO, that recovers xNES from abstract principles.