Simulations are used in science and industry to predict the performance of technical systems. Adjoint derivatives of these simulations can reveal the sensitivity of the system performance to changes in design or operating conditions, and are increasingly used in shape optimisation and uncertainty quantification. Algorithmic differentiation (AD) by source-transformation is an efficient method to compute such derivatives. AD requires an analysis of the computation and its data flow to produce efficient adjoint code. One important step is the activity analysis that detects operations that need to be differentiated. An improved activity analysis is investigated in this thesis that simplifies build procedures for certain adjoint programs, and is demonstrated to improve the speed of an adjoint fluid dynamics solver. The method works by allowing a context-dependent analysis of routines. The ongoing trend towards multi- and many-core architectures such as the Intel XeonPhi is creating challenges for AD. Two novel approaches are presented that replicate the parallelisation of a program in its corresponding adjoint program. The first approach detects loops that naturally result in a parallelisable adjoint loop, while the second approach uses loop transformation and the aforementioned context-dependent analysis to enforce parallelisable data access in the adjoint loop. A case study shows that both approaches yield adjoints that are as scalable as their underlying primal programs. Adjoint computations are limited by their memory footprint, particularly in unsteady simulations, for which this work presents incomplete checkpointing as a method to reduce memory usage at the cost of a slight reduction in accuracy. Finally, convergence of iterative linear solvers is discussed, which is especially relevant on accelerator cards, where single precision floating point numbers are frequently used and the choice of solvers is limited by the small memory size. Some problems that are particular to adjoint computations are discussed.

In dieser Arbeit werden Techniken beschrieben, die es erlauben (höhere) Ableitungen und Taylorapproximationen solcher Computerprogramme effizient zu berechnen. Auch inbesondere dann, wenn die Programme Algorithmen der numerischen linearen Algebra (NLA) enthalten. Im Gegensatz zur traditionellen algorithmischen Differentiation (AD), bei der die zugrunde liegenden Algorithmen um zusätzliche Befehlere erweitert werden, sind in dieser Arbeit die Zerlegungen durch definierende Gleichungen charakterisiert. Basierend auf den definierenden Gleichungen werden Strukturausnutzende Algorithmen hergeleitet. Genauer, neuartige Algorithmen für die Propagation von Taylorpolynomen durch die QR, Cholesky und reell-symmetrischen Eigenwertzerlegung werden präsentiert. Desweiteren werden Algorithmen für den Rückwärtsmodus der AD hergeleitet, welche im Wesentlichen nur die Faktoren der Zerlegungen benötigen. Im Vergleich zum traditionellen Ansatz, bei dem alle Zwischenergebnisse gespeichert werden, ist dies eine Reduktion von O(N^3) zu O(N^2) für Algorithmen mit O(N^3) Komplexität. N ist hier die Größe der Matrix. Zusätzlich kann bestehende, hoch-optimierte Software verwendet werden. Ein Laufzeitvergleich zeigt, dass dies im Vergleich zum traditionellen Ansatz zu einer Beschleunigung in der Größenordnung 100 führen kann. Da die NLA Funktionen als Black Box betrachtet werden, ist desweiteren auch der Berechnungsgraph um Größenordnungen kleiner. Dies bedeutet, dass Software, welche Operator Overloading benutzt, weniger Overhead hervorruft und auch weniger Speicher benötigt. / This thesis provides a framework for the evaluation of first and higher-order derivatives and Taylor series expansions through large computer programs that contain numerical linear algebra (NLA) functions. It is a generalization of traditional algorithmic differentiation (AD) techniques in that NLA functions are regarded as black boxes where the inputs and outputs are related by defining equations. Based on the defining equations, structure-exploiting algorithms are derived. More precisely, novel algorithms for the propagation of Taylor polynomials through the QR, Cholesky,- and real-symmetric eigenvalue decomposition are shown. Recurrences for the reverse mode of AD, which require essentially only the returned factors of the decomposition, are also derived. Compared to the traditional approach where all intermediates of an algorithm are stored, this is a reduction from O(N^3) to O(N^2) for algorithms with O( N^3) complexity. N denotes the matrix size. The derived algorithms make it possible to use existing high-performance implementations. A runtime comparison shows that the treatment of NLA functions as atomic can be more than one order of magnitude faster than an automatic differentiation of the underlying algorithm. Furthermore, the computational graph is orders of magnitudes smaller. This reduces the additional memory requirements, as well as the overhead, of operator overloading techniques to a fraction.

Le mode adjoint de la Différentiation Algorithmique (DA) est particulièrement intéressant pour le calcul des gradients. Cependant, ce mode utilise les valeurs intermédiaires de la simulation d'origine dans l'ordre inverse à un coût qui augmente avec la longueur de la simulation. La DA cherche des stratégies pour réduire ce coût, par exemple en profitant de la structure du programme donné. Dans ce travail, nous considérons d'une part le cas des boucles à point-fixe pour lesquels plusieurs auteurs ont proposé des stratégies adjointes adaptées. Parmi ces stratégies, nous choisissons celle de B. Christianson. Nous spécifions la méthode choisie et nous décrivons la manière dont nous l'avons implémentée dans l'outil de DA Tapenade. Les expériences sur une application de taille moyenne montrent une réduction importante de la consommation de mémoire. D'autre part, nous étudions le checkpointing dans le cas de programmes parallèles MPI avec des communications point-à-point. Nous proposons des techniques pour appliquer le checkpointing à ces programmes. Nous fournissons des éléments de preuve de correction de nos techniques et nous les expérimentons sur des codes représentatifs. Ce travail a été effectué dans le cadre du projet européen ``AboutFlow'' / The adjoint mode of Algorithmic Differentiation (AD) is particularly attractive for computing gradients. However, this mode needs to use the intermediate values of the original simulation in reverse order at a cost that increases with the length of the simulation. AD research looks for strategies to reduce this cost, for instance by taking advantage of the structure of the given program. In this work, we consider on one hand the frequent case of Fixed-Point loops for which several authors have proposed adapted adjoint strategies. Among these strategies, we select the one introduced by B. Christianson. We specify further the selected method and we describe the way we implemented it inside the AD tool Tapenade. Experiments on a medium-size application shows a major reduction of the memory needed to store trajectories. On the other hand, we study checkpointing in the case of MPI parallel programs with point-to-point communications. We propose techniques to apply checkpointing to these programs. We provide proof of correctness of our techniques and we experiment them on representative CFD codes

The master thesis is focused on how a local volatility surfaces can be extracted by optimization with respectto smoothness and price error. The pricing is based on utility based pricing, and developed to be set in arisk neutral pricing setting. The pricing is done in a discrete multinomial recombining tree, where the timeand price increments optionally can be equidistant. An interpolation algorithm is used if the option that shallbe priced is not matched in the tree discretization. Power utility functions are utilized, where the log-utilitypreference is especially studied, which coincides with the (Kelly) portfolio that systematically outperforms anyother portfolio. A fine resolution of the discretization is generally a property that is sought after, thus a seriesof derivations for the implementation are done to restrict the computational encumbrance and thus allow finer discretization. The thesis is mainly focused on the derivation of the method rather than finding optimal parameters thatgenerate the local volatility surfaces. The method has shown that smooth surfaces can be extracted, whichconsider market prices. However, due to lacking available interest and dividend data, the pricing error increasessymmetrically for longer option maturities. However, the method shows exponential convergence and robustnessto different initial (flat) volatilities for the optimization initiation. Given an optimal smooth local volatility surface, an arbitrary payoff function can then be used to price thecorresponding option, which could be path-dependent, such as barrier options. However, only vanilla optionswill be considered in this thesis. Finally, we find that the developed

When modelling risk in the banking book, a simple capital level approach can fail to capture the interactions between different risk measures or risk classes since they are modelled separately. In this thesis we propose a model for projecting the book value of a run-off balance sheet portfolio of fixed and variable rate loans, while also calculating net interest income, economic value of equity, capital requirement and capital cost within the same model. Using adjoint algorithmic differentiation, we also retrieve the sensitivities of each measure and the balance sheet towards a term structure of zero rates, for the lifetime of the portfolio. The model is an attempt at a holistic approach to modelling interest rate risk in the banking book, and its design allows for extensions to other financial risk classes such as credit risk and liquidity risk. / Vid modellering av risker i bankboken kan en enkel kapitalmodell misslyckas med att fånga interaktionerna mellan olika riskmått eller riskklasser eftersom de modelleras separat. I den här masteruppsatsen föreslår vi en modell för att projicera balansräkningen framåt i tiden för en portfölj av lån med fast och rörlig ränta, och beräknar samtidigt Räntenetto, Ekonomiskt Värde av Eget Kapital, kapitalkrav och kapitalkostnad inom samma modell. Med hjälp av adjoint algoritmisk derivering beräknar vi även känsligheterna för varje mått samt bokfört värde av portföljen mot en terminstruktur av nollräntor under portföljens livstid. Modellen är ett försök att holistisk modellera ränterisk i bankboken, och dess utformning möjliggör utvidgning till andra finansiella riskklasser som kreditrisk och likviditetsrisk.

Zuweilen wird gefördertes Erdgas als eine Brückentechnologie noch eine Weile erhalten bleiben, aber unsere Gasnetzinfrastruktur hat auch in einer Ära post-fossiler Brennstoffe eine Zukunft, um Klima-neutral erzeugtes Methan, Ammoniak oder Wasserstoff zu transportieren. Damit die Dispatcher der Zukunft, in einer sich fortwährend dynamisierenden Marktsituation, mit sich beständig wechselnden Kleinstanbietern, auch weiterhin einen sicheren Gasnetzbetrieb ermöglichen und garantieren können, werden sie auf moderne, schnelle Simulations- sowie performante Optimierungstechnologie angewiesen sein. Der Schlüssel dazu liegt in einem besseren Verständnis zur numerischen Behandlung nicht differenzierbarer Funktionen und diese Arbeit möchte einen Beitrag hierzu leisten. Wir werden stückweise differenzierbare Funktionen in sog. Abs-Normalen Form betrachten. Durch einen Prozess, der Abs-Linearisierung genannt wird, können wir stückweise lineare Approximationsmodelle erster Ordnung, mittels Techniken der algorithmischen Differentiation erzeugen. Jene Modelle können über Matrizen und Vektoren mittels gängiger Software-Bibliotheken der numerischen linearen Algebra auf Computersystemen ausgedrückt, gespeichert und behandelt werden. Über die Generalisierung der Formel von Faà di Bruno können auch Splinefunktionen höherer Ordnung generiert werden, was wiederum zu Annäherungsmodellen mit besserer Güte führt. Darauf aufbauend lassen sich gemischte Taylor-Kollokationsmethoden, darunter die mit Ordnung zwei konvergente generalisierte Trapezmethode, zur Integration von Gasnetzen, in Form von nicht glatten Algebro-Differentialgleichungssystemen, definieren. Numerische Experimente demonstrieren das Potential. Da solche implizite Integratoren auch nicht lineare und in unserem Falle zugleich auch stückweise differenzierbare Gleichungssysteme erzeugen, die es als Unterproblem zu lösen gilt, werden wir uns auch die stückweise differenzierbare, sowie die stückweise lineare Newtonmethode betrachten. / As of yet natural gas will remain as a bridging technology, but our gas grid infrastructure does have a future in a post-fossil fuel era for the transportation of carbon-free produced methane, ammonia or hydrogen. In order for future dispatchers to continue to enable and guarantee safe gas network operations in a continuously changing market situation with constantly switching micro-suppliers, they will be dependent on modern, fast simulation as well as high-performant optimization technology. The key to such a technology resides in a better understanding of the numerical treatment of non-differentiable functions and this work aims to contribute here. We will consider piecewise differentiable functions in so-called abs-normal form. Through a process called abs-linearization, we can generate piecewise linear approximation models of order one, using techniques of algorithmic differentiation. Those models can be expressed, stored and treated numerically as matrices and vectors via common software libraries of numerical linear algebra. Generalizing the Faà di Bruno's formula yields higher order spline functions, which in turn leads to even higher order approximation models. Based on this, mixed Taylor-Collocation methods, including the generalized trapezoidal method converging with an order of two, can be defined for the integration of gas networks represented in terms of non-smooth system of differential algebraic equations. Numerical experiments will demonstrate the potential. Since those implicit integrators do generate non-linear and, in our case, piecewise differentiable systems of equations as sub-problems, it will be necessary to consider the piecewise differentiable, as well as the piecewise linear Newton method in advance.

As wind energy technology continues to mature, farm sizes grow and wind farm layout design becomes more difficult, in part due to the number of design variables and constraints. Wind farm layout optimization is typically approached using gradient-free methods because of the highly multi-modal shape of the wind farm layout design space. Gradient-free method performance generally degrades with increasing problem size, making it difficult to find optimal layouts for larger wind farms. However, gradient-based optimization methods can effectively and efficiently solve large-scale problems with many variables and constraints. To pave the way for effective and efficient wind farm layout optimization for large-scale wind farms, we have worked to overcome the primary barriers to applying gradient-based optimization to wind farm layout optimization. To improve model/algorithm compatibility, we adjusted wake and wind farm models, adding more realistic curvature and smoothness to enable optimization algorithms to travel through areas in the design space where they had previously gotten stuck. We reduced the number of function calls required for gradient-based wind farm layout optimization by over three orders of magnitude for large farms by using algorithmic differentiation to compute derivatives. We reduced the multi-modality of the wind farm layout design space using wake expansion continuation (WEC). We developed WEC to work with existing optimization algorithms, enabling them to get out of local optima while remaining fully gradient-based. Across four case studies, WEC found results with lower wake loss, on average, than the other methods we tested. To resolve concerns about optimization algorithms exploiting model inaccuracies, we compared the initial and optimized layouts to large-eddy simulation (LES) results. The simple models predicted an AEP improvement of 7.7% for a low-TI case, and LES predicted 9.3%. For a high-TI case, the simple models predicted a 10.0% improvement in AEP and LES predicted 10.7%. To resolve uncertainty regarding relative solution quality for gradient-based and gradient-free methods, we collaborated with seven organizations to compare eight optimization methods. Each method was managed by researchers experienced with them. All methods found solutions of similar quality, with optimized wake loss between 15.48 % and 15.70 %. WEC with SNOPT was the only purely gradient-based method included and found the third-to-best solution.