A study was performed to compare two different methods for generating surrogate models for the analysis and design of airfoils. Initial research was performed to compare the accuracy of surrogate models for predicting the lift and drag of an airfoil with data collected from highidelity simulations using a modern CFD code along with lower-order models using a panel code. This was followed by an evaluation of the Class Shape Trans- formation (CST) method for parameterizing airfoil geometries as a prelude to the use of surrogate models for airfoil design optimization and the implementation of software to use CST to modify airfoil shapes as part of the airfoil design process. Optimization routines were coupled with surrogate modeling techniques to study the accuracy and efficiency of the surrogate models to produce optimal airfoil shapes. Finally, the results of the current research are summarized, and suggestions are made for future research.

No description available.

No description available.

Time delays are an inherent part of real-world systems. Besides the apparent slowing of the system, these time delays often cause destabilization in otherwise stable systems, and perhaps even more unexpectedly, can stabilize an unstable system. Here, we propose the Stochastic Time-Delayed Adaptation as a method for improving optimization on certain high-dimensional surfaces, which simply wraps a known optimizer --such as the Adam optimizer-- and is able to add a variety of time-delays. We begin by exploring time delays on certain gradient-based optimization methods and their affect on the optimizer's convergence properties. These optimizers include the standard gradient descent method and the more recent Adam Optimizer, where the latter is commonly used in neural networks for deep learning. To begin to describe the effect of time-delays on these methods, we use the theory of intrinsic stability. It has been shown that a system that possesses the property of intrinsic stability (a stronger form of global stability) will maintain its stability when subject to any time delays, e.g., constant, periodic, stochastic, etc. In feasible cases, we find relevant conditions under which the optimization method adapted with time delays is intrinsically stable and therefore converges to the system's minimal value. Finally, we examine the optimizer's performance using common optimizer performance metrics. This includes the number of steps an algorithm takes to converge and also the final loss value in relation to the global minimum of the loss function. We test these outcomes using various adaptations of the Adam optimizer on multiple common test optimization functions, which are designed to be difficult for vanilla optimizer methods. We show that the Stochastic Time-Delayed Adaptation can greatly improve an optimizer's ability to find a global minimum of a complex loss function.

Many novel graph neural network models have reported an impressive performance on benchmark dataset, but the theory behind these networks is still being developed. In this thesis, we study the trajectory of Gradient descent (GD) and Stochastic gradient descent (SGD) in the loss landscape of Graph neural networks by replicating Xing et al. [1] study for feed-forward networks. Furthermore, we empirically examine if the training process could be accelerated by an optimization algorithm inspired from Stochastic gradient Langevin dynamics and what effect the topology of the graph has on the convergence of GD by perturbing its structure. We find that the loss landscape is relatively flat and that SGD does not encounter any significant obstacles during its propagation. The noise-induced gradient appears to aid SGD in finding a stationary point with desirable generalisation capabilities when the learning rate is poorly optimized. Additionally, we observe that the topological structure of the graph plays a part in the convergence of GD but further research is required to understand how. / Många nya grafneurala nätverk har visat imponerande resultat på existerande dataset, dock är teorin bakom dessa nätverk fortfarande under utveckling. I denna uppsats studerar vi banor av gradientmetoden (GD) och den stokastiska gradientmetoden (SGD) i lösningslandskapet till grafiska faltningsnätverk genom att replikera studien av feed-forward nätverk av Xing et al. [1]. Dessutom undersöker vi empiriskt om träningsprocessen kan accelereras genom en optimeringsalgoritm inspirerad av Stokastisk gradient Langevin dynamik, samt om grafens topologi har en inverkan på konvergensen av GD genom att ändra strukturen. Vi ser att lösningslandskapet är relativt plant och att bruset inducerat i gradienten verkar hjälpa SGD att finna stabila stationära punkter med önskvärda generaliseringsegenskaper när inlärningsparametern har blivit olämpligt optimerad. Dessutom observerar vi att den topologiska grafstrukturen påverkar konvergensen av GD, men det behövs mer forskning för att förstå hur.

Os sistemas elétricos de potência são frequentemente submetidos a perturbações causadas, por exemplo, por um aumento súbito de carga ou por um curto-circuito em uma linha de transmissão. Estas perturbações podem gerar oscilações eletromecânicas no sistema, uma vez que a velocidade dos geradores oscila. Para reduzir tais oscilações, controladores de sistema de potência são utilizados sendo, os mais comuns, controladores do tipo PSS (Power System Stabilizer). Porém, em alguns sistemas, somente o emprego de PSSs não é suficiente para garantir um nível mínimo satisfatório de amortecimento, sendo necessário recorrer a outros tipos de controladores. Desta forma, controladores do tipo FACTS (Flexible Alternating Current Transmission System), principalmente o TCSC (Thyristor Controlled Series Capacitor) se tornaram uma alternativa atraente no auxílio ao amortecimento dos sistemas de potência. O controle do TCSC é feito por um controlador POD (Power Oscilation Damper) que é empregado como controle suplementar do dispositivo. No entanto, apenas o emprego dos controladores não garante um bom amortecimento, fazendo-se necessária uma boa sintonia dos mesmos. A sintonia destes controladores é, na maioria das vezes, feita de forma manual, ou seja, usando o método de tentativa e erro, podendo demandar um tempo relativamente elevado, mesmo que feita por um engenheiro experiente. Para evitar que o projetista dispense tempo procurando uma sintonia adequada para os controladores, métodos automáticos de sintonia vêm sendo estudados. Tais métodos têm como principal vantagem a sintonia dos controladores considerando vários pontos de operação do sistema simultaneamente, fazendo com que o controlador seja robusto para variações em seu ponto de operação nominal. Os métodos automáticos de sintonia utilizam métodos de otimização local ou métodos de otimização global. Os métodos de busca local têm a rapidez como principal vantagem, porém podem não convergir para um valor satisfatório de amortecimento estipulado pelo projetista. Os métodos de busca global, em grande parte das vezes, convergem para um valor de amortecimento solicitado pelo projetista, porém em um tempo elevado. Este trabalho propõe um método de sintonia dos controladores utilizando uma estrutura híbrida, ou seja, algoritmos de busca global juntamente com algoritmos de busca local. Primeiramente uma busca global é feita pelo algoritmo até que um critério de parada, definido pelo projetista, seja alcançado (geralmente um valor de amortecimento mínimo para o sistema). Assim, os parâmetros dos controladores sintonizados pela busca global serão entrada de um método de busca local. O algoritmo de busca local tende a refinar a sintonia dos controladores aumentando, assim, o amortecimento do sistema até um valor especificado pelo projetista. Neste trabalho a busca global é realizada por um algoritmo genético enquanto que a busca local é feita por um algoritmo baseado no gradiente descendente da função objetivo (neste caso o amortecimento). As principais vantagens do algoritmo proposto são a diminuição do tempo de sintonia e o esforço computacional, se comparado a métodos de busca global, verificadas nos resultados do trabalho. / Electric Power Systems are constantly subjected to perturbations, which can be caused for several different reasons, e.g., due to a sudden load increase or a short circuit in a transmission line. These perturbations can induce electromechanical oscillations in the power system, since the angular speed of the generators oscillates. To reduce such oscillations, power system controllers are used, and the most common ones are the PSSs (Power System Stabilizers). In some systems, however, the usage of PSSs is not sufficient to guarantee a satisfactory level for the minimum damping, being necessary the usage of other types of controllers. Hence, FACTS (Flexible Alternating Current Transmission System) controllers, specially the TCSC (Thyristor Controlled Series Capacitor), became an attractive alternative to enhance the damping of electric power systems. The TCSC control action is performed by a POD (Power Oscillation Damper) controller, which is a supplementary control function of the device. However, it is not only the usage of such controllers that guarantees a sufficient damping, but also a good tuning of their parameters. The tuning of such controllers is ordinarily performed manually, using a trial-and-error method, which can last for a long time, even for experienced engineers. To facilitate the designers work in the tuning of the controllers parameters, automatic tuning methods are being studied. Such methods have the main advantage of considering several operating points of the system simultaneously, yielding a robust controller regarding variations in its nominal operating point. The aforementioned automatic tuning methods use local optimization methods or global optimization methods. The local optimization methods have the speed as the main advantage, but they can have convergence issues in the search for the minimum satisfactory damping threshold desired by the designer. The global optimization methods, on the other hand, ordinarily converge for the desired minimum damping threshold, but with large convergence times. This work proposes a controller tuning method using a hybrid structure, i.e., global search methods with local search methods. Initially, a global search is performed by the algorithm until a stop criteria is met, as defined by the designer (usually a minimum damping for the system). Thus, the controller parameters tuned by the global search method are the input values of a local search method. The local search algorithm actually refines the controllers tuning, increasing the system damping to the value defined by the designer. In this work the global search is performed by a genectic algorithm while the local search is performed by an algorithm based in the gradient descent of objective function (damping in this case). The greatest advantages of the proposed algorithm are the possible decrease in computational time and effort, when compared to global search methods, verified in the work results.

Os sistemas elétricos de potência são frequentemente submetidos a perturbações causadas, por exemplo, por um aumento súbito de carga ou por um curto-circuito em uma linha de transmissão. Estas perturbações podem gerar oscilações eletromecânicas no sistema, uma vez que a velocidade dos geradores oscila. Para reduzir tais oscilações, controladores de sistema de potência são utilizados sendo, os mais comuns, controladores do tipo PSS (Power System Stabilizer). Porém, em alguns sistemas, somente o emprego de PSSs não é suficiente para garantir um nível mínimo satisfatório de amortecimento, sendo necessário recorrer a outros tipos de controladores. Desta forma, controladores do tipo FACTS (Flexible Alternating Current Transmission System), principalmente o TCSC (Thyristor Controlled Series Capacitor) se tornaram uma alternativa atraente no auxílio ao amortecimento dos sistemas de potência. O controle do TCSC é feito por um controlador POD (Power Oscilation Damper) que é empregado como controle suplementar do dispositivo. No entanto, apenas o emprego dos controladores não garante um bom amortecimento, fazendo-se necessária uma boa sintonia dos mesmos. A sintonia destes controladores é, na maioria das vezes, feita de forma manual, ou seja, usando o método de tentativa e erro, podendo demandar um tempo relativamente elevado, mesmo que feita por um engenheiro experiente. Para evitar que o projetista dispense tempo procurando uma sintonia adequada para os controladores, métodos automáticos de sintonia vêm sendo estudados. Tais métodos têm como principal vantagem a sintonia dos controladores considerando vários pontos de operação do sistema simultaneamente, fazendo com que o controlador seja robusto para variações em seu ponto de operação nominal. Os métodos automáticos de sintonia utilizam métodos de otimização local ou métodos de otimização global. Os métodos de busca local têm a rapidez como principal vantagem, porém podem não convergir para um valor satisfatório de amortecimento estipulado pelo projetista. Os métodos de busca global, em grande parte das vezes, convergem para um valor de amortecimento solicitado pelo projetista, porém em um tempo elevado. Este trabalho propõe um método de sintonia dos controladores utilizando uma estrutura híbrida, ou seja, algoritmos de busca global juntamente com algoritmos de busca local. Primeiramente uma busca global é feita pelo algoritmo até que um critério de parada, definido pelo projetista, seja alcançado (geralmente um valor de amortecimento mínimo para o sistema). Assim, os parâmetros dos controladores sintonizados pela busca global serão entrada de um método de busca local. O algoritmo de busca local tende a refinar a sintonia dos controladores aumentando, assim, o amortecimento do sistema até um valor especificado pelo projetista. Neste trabalho a busca global é realizada por um algoritmo genético enquanto que a busca local é feita por um algoritmo baseado no gradiente descendente da função objetivo (neste caso o amortecimento). As principais vantagens do algoritmo proposto são a diminuição do tempo de sintonia e o esforço computacional, se comparado a métodos de busca global, verificadas nos resultados do trabalho. / Electric Power Systems are constantly subjected to perturbations, which can be caused for several different reasons, e.g., due to a sudden load increase or a short circuit in a transmission line. These perturbations can induce electromechanical oscillations in the power system, since the angular speed of the generators oscillates. To reduce such oscillations, power system controllers are used, and the most common ones are the PSSs (Power System Stabilizers). In some systems, however, the usage of PSSs is not sufficient to guarantee a satisfactory level for the minimum damping, being necessary the usage of other types of controllers. Hence, FACTS (Flexible Alternating Current Transmission System) controllers, specially the TCSC (Thyristor Controlled Series Capacitor), became an attractive alternative to enhance the damping of electric power systems. The TCSC control action is performed by a POD (Power Oscillation Damper) controller, which is a supplementary control function of the device. However, it is not only the usage of such controllers that guarantees a sufficient damping, but also a good tuning of their parameters. The tuning of such controllers is ordinarily performed manually, using a trial-and-error method, which can last for a long time, even for experienced engineers. To facilitate the designers work in the tuning of the controllers parameters, automatic tuning methods are being studied. Such methods have the main advantage of considering several operating points of the system simultaneously, yielding a robust controller regarding variations in its nominal operating point. The aforementioned automatic tuning methods use local optimization methods or global optimization methods. The local optimization methods have the speed as the main advantage, but they can have convergence issues in the search for the minimum satisfactory damping threshold desired by the designer. The global optimization methods, on the other hand, ordinarily converge for the desired minimum damping threshold, but with large convergence times. This work proposes a controller tuning method using a hybrid structure, i.e., global search methods with local search methods. Initially, a global search is performed by the algorithm until a stop criteria is met, as defined by the designer (usually a minimum damping for the system). Thus, the controller parameters tuned by the global search method are the input values of a local search method. The local search algorithm actually refines the controllers tuning, increasing the system damping to the value defined by the designer. In this work the global search is performed by a genectic algorithm while the local search is performed by an algorithm based in the gradient descent of objective function (damping in this case). The greatest advantages of the proposed algorithm are the possible decrease in computational time and effort, when compared to global search methods, verified in the work results.

Spectrally sparse signals arise in many applications of signal processing. A spectrally sparse signal is a mixture of a few undamped or damped complex sinusoids. An important problem from practice is to reconstruct such a signal from partial time domain samples. Previous convex methods have the drawback that the computation and storage costs do not scale well with respect to the signal length. This common drawback restricts their applicabilities to large and high-dimensional signals. The reconstruction of a spectrally sparse signal from partial samples can be formulated as a low-rank Hankel matrix completion problem. We develop two fast and provable non-convex solvers, FIHT and PGD. FIHT is based on Riemannian optimization while PGD is based on Burer-Monteiro factorization with projected gradient descent. Suppose the underlying spectrally sparse signal is of model order r and length n. We prove that O(r^2log^2(n)) and O(r^2log(n)) random samples are sufficient for FIHT and PGD respectively to achieve exact recovery with overwhelming probability. Every iteration, the computation and storage costs of both methods are linear with respect to signal length n. Therefore they are suitable for handling spectrally sparse signals of large size, which may be prohibited for previous convex methods. Extensive numerical experiments verify their recovery abilities as well as computation efficiency, and also show that the algorithms are robust to noise and mis-specification of the model order. Comparing the two solvers, FIHT is faster for easier problems while PGD has a better recovery ability.

An analog circuit for beamforming in a mobile Ku band satellite TV antenna array has been implemented. The circuit performs continuous-time gradient descent using simultaneous perturbation gradient estimation. Simulations were performed using Agilent ADS circuit simulator. Field tests were performed in a realistic scenario using a satellite signal. The results were comparable to the simulation predictions and to results obtained using a digital implementation of a similar stochastic approximation algorithm.

An analog circuit for beamforming in a mobile Ku band satellite TV antenna array has been implemented. The circuit performs continuous-time gradient descent using simultaneous perturbation gradient estimation. Simulations were performed using Agilent ADS circuit simulator. Field tests were performed in a realistic scenario using a satellite signal. The results were comparable to the simulation predictions and to results obtained using a digital implementation of a similar stochastic approximation algorithm.