Thesis (M.S.)--Worcester Polytechnic Institute. / Keywords: bicgstab, cgs, gmres, NITSOL, Newtons Method, nonlinear systems. Includes bibliographical references (p. 53).

Thesis (M.S.)--Worcester Polytechnic Institute. / Keywords: Newton; globalized; inexact Newton. Includes bibliographical references (p. 61).

Resumo: Nesse trabalho propõe-se uma adaptação do método barreira penalidade quasi-Newton apresentado por P. Armand em 2003, para a resolução do problema do Fluxo de Potência Ótimo (FPO). Este método é denominado de método da função langrangiana barreira penalidade adaptada. Neste método as restrições de desigualdade são transformadas em igualdade pelo uso de variáveis de folga positivas. Estas variáveis são relaxadas, utilizando-se variáveis positivas, as quais, são incorporadas na função objetivo através de um termo de penalização. O novo problema restrito é então transformado em irrestrito associando a uma função lagrangiana às restrições de igualdade e uma função barreira penalidade às restrições de desigualdade. o algoritmo é composto por um ciclo interno e um externo. No ciclo interno é utilizado um método quasi-Newton para o cálculo das direções de busca e é determinado o tamanho do passo. No ciclo externo os parâmetros de barreira e penalidade são atualizados através de regras pré-definidas até que as condições de KKT sejam satisfeitas. Testes computacionais foram realizados utilizando problemas matemáticos e o problema de FPO, os quais demonstram a eficiência da adaptação proposta / Abstract: This work proposes an adaptation of the quasi-Newton penalty barrier method presented by P. Armand in 2003. for the solution of the Optimal Power Flow (OPD) problem. This method is called method adapted penalty barrier lagrangian function. In this method the inequalities constraint are transformed in equality by adding non-negative slack variable. These variables are relaxed by positive auxiliary variables which are incorporated in the objective function through a penalty term. The new constraint problem is transformed in unconstraint by associating an lagrangian function for handling the equality constraint and an penalty barrier function for treating the inequality constraints. The algorithm is composed by an internal and external cycle. In the interanal cycle is used the quasi-Newton method to determine the search directions and the step size is calculated. In the external cycle the barrier parameters are updated through predefined rules until the KKT conditions are satisfied. Computational tests were accomplished using mathematical problems and the OPF problem which demonstrate the efficiency of the propose adaptation / Orientador: Edméa Cássia Baptista / Coorientador: Vanusa Alves de Sousa / Banca: Geraldo Roberto Martins da Costa / Banca: Antonio Roberto Balbo / Mestre

NITSOL: A Newton Iterative Solver for Nonlinear Systems describes an algorithm for solving nonlinear systems. Michael Pernice and Homer F. Walker, the authors of the paper NITSOL [3], implemented this algorithm in FORTRAN. The goal of the project has been to use the modern and robust language MATLAB to implement the NITSOL algorithm. In this paper, the main mathematical and algorithmic background for understanding NITSOL are described, and a user guide is included outlining how to use the MATLAB implementation of NITSOL. A nonlinear system example problem, the 2D Bratu problem, and the solution obtained by MATLAB NITSOL's are also included.

Newton's method is at the core of many algorithms used for solving nonlinear equations. A globalized Newton method is an implementation of Newton's method augmented with ``globalization procedures' intended to enhance the likelihood of convergence to a solution from an arbitrary initial guess. A Newton-GMRES method is an implementation of Newton's method in which the iterative linear algebra method GMRES is used to solve approximately the linear system that characterizes the Newton step. A globalized Newton-GMRES method combines both globalization procedures and the GMRES scheme to develop robust and efficient algorithms for solving nonlinear equations. The aim of this project is to describe the development of some globalized Newton-GMRES methods and to compare their performances on a few benchmark fluid flow problems.

abstract: This thesis presents a family of adaptive curvature methods for gradient-based stochastic optimization. In particular, a general algorithmic framework is introduced along with a practical implementation that yields an efficient, adaptive curvature gradient descent algorithm. To this end, a theoretical and practical link between curvature matrix estimation and shrinkage methods for covariance matrices is established. The use of shrinkage improves estimation accuracy of the curvature matrix when data samples are scarce. This thesis also introduce several insights that result in data- and computation-efficient update equations. Empirical results suggest that the proposed method compares favorably with existing second-order techniques based on the Fisher or Gauss-Newton and with adaptive stochastic gradient descent methods on both supervised and reinforcement learning tasks. / Dissertation/Thesis / Masters Thesis Computer Science 2019

Neural network training algorithms have always suffered from the problem of local minima. The advent of natural gradient algorithms promised to overcome this shortcoming by finding better local minima. However, they require additional training parameters and computational overhead. By using a new formulation for the natural gradient, an algorithm is described that uses less memory and processing time than previous algorithms with comparable performance.

Made available in DSpace on 2014-06-11T19:22:34Z (GMT). No. of bitstreams: 0 Previous issue date: 2011-08-17Bitstream added on 2014-06-13T18:49:37Z : No. of bitstreams: 1 campanha_ps_me_bauru.pdf: 713465 bytes, checksum: 80f1a0cfec7a9f0dda4e30ae9f1786ab (MD5) / Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) / Nesse trabalho propõe-se uma adaptação do método barreira penalidade quasi-Newton apresentado por P. Armand em 2003, para a resolução do problema do Fluxo de Potência Ótimo (FPO). Este método é denominado de método da função langrangiana barreira penalidade adaptada. Neste método as restrições de desigualdade são transformadas em igualdade pelo uso de variáveis de folga positivas. Estas variáveis são relaxadas, utilizando-se variáveis positivas, as quais, são incorporadas na função objetivo através de um termo de penalização. O novo problema restrito é então transformado em irrestrito associando a uma função lagrangiana às restrições de igualdade e uma função barreira penalidade às restrições de desigualdade. o algoritmo é composto por um ciclo interno e um externo. No ciclo interno é utilizado um método quasi-Newton para o cálculo das direções de busca e é determinado o tamanho do passo. No ciclo externo os parâmetros de barreira e penalidade são atualizados através de regras pré-definidas até que as condições de KKT sejam satisfeitas. Testes computacionais foram realizados utilizando problemas matemáticos e o problema de FPO, os quais demonstram a eficiência da adaptação proposta / This work proposes an adaptation of the quasi-Newton penalty barrier method presented by P. Armand in 2003. for the solution of the Optimal Power Flow (OPD) problem. This method is called method adapted penalty barrier lagrangian function. In this method the inequalities constraint are transformed in equality by adding non-negative slack variable. These variables are relaxed by positive auxiliary variables which are incorporated in the objective function through a penalty term. The new constraint problem is transformed in unconstraint by associating an lagrangian function for handling the equality constraint and an penalty barrier function for treating the inequality constraints. The algorithm is composed by an internal and external cycle. In the interanal cycle is used the quasi-Newton method to determine the search directions and the step size is calculated. In the external cycle the barrier parameters are updated through predefined rules until the KKT conditions are satisfied. Computational tests were accomplished using mathematical problems and the OPF problem which demonstrate the efficiency of the propose adaptation

Includes bibliographical references / In this thesis, the development of a strongly coupled, partitioned fluid-structure interactions (FSI) solver is outlined. Well established methods are analysed and new methods are proposed to provide robust, accurate and efficient FSI solutions. All the methods introduced and analysed are primarily geared towards the solution of incompressible, transient FSI problems, which facilitate the use of black-box sub-domain field solvers. In the first part of the thesis, radial basis function (RBF) interpolation is introduced for interface information transfer. RBF interpolation requires no grid connectivity information, and therefore presents an elegant means by which to transfer information across a non-matching and non-conforming interface to couple finite element to finite volume based discretisation schemes. The transfer scheme is analysed, with particular emphasis on a comparison between consistent and conservative formulations. The primary aim is to demonstrate that the widely used conservative formulation is a zero order method. Furthermore, while the consistent formulation is not provably conservative, it yields errors well within acceptable levels and converges within the limit of mesh refinement. A newly developed multi-vector update quasi-Newton (MVQN) method for implicit coupling of black-box partitioned solvers is proposed. The new coupling scheme, under certain conditions, can be demonstrated to provide near Newton-like convergence behaviour. The superior convergence properties and robust nature of the MVQN method are shown in comparison to other well-known quasi-Newton coupling schemes, including the least squares reduced order modelling (IBQN-LS) scheme, the classical rank-1 update Broyden's method, and fixed point iterations with dynamic relaxation. Partitioned, incompressible FSI, based on Dirichlet-Neumann domain decomposition solution schemes, cannot be applied to problems where the fluid domain is fully enclosed. A simple example often provided in the literature is that of balloon inflation with a prescribed inflow velocity. In this context, artificial compressibility (AC) will be shown to be a useful method to relax the incompressibility constraint, by including a source term within the fluid continuity equation. The attractiveness of AC stems from the fact that this source term can readily be added to almost any fluid field solver, including most commercial solvers. AC/FSI is however limited in the range of problems it can effectively be applied to. To this end, the combination of the newly developed MVQN method with AC/FSI is proposed. In so doing, the AC modified fluid field solver can continue to be treated as a black-box solver, while the overall robustness and performance are significantly improved. The study concludes with a demonstration of the modularity offered by partitioned FSI solvers. The analysis of the coupled environment is extended to include steady state FSI, FSI with free surfaces and an FSI problem with solid-body contact.

Variational methods and Partial Differential Equations (PDEs) have been extensively employed for the mathematical formulation of a myriad of problems describing physical phenomena such as heat propagation, thermodynamic transformations and many more. In imaging, PDEs following variational principles are often considered. In their general form these models combine a regularisation and a data fitting term, balancing one against the other appropriately. Total variation (TV) regularisation is often used due to its edgepreserving and smoothing properties. In this thesis, we focus on the design of TV-based models for several different applications. We start considering PDE models encoding higher-order derivatives to overcome wellknown TV reconstruction drawbacks. Due to their high differential order and nonlinear nature, the computation of the numerical solution of these equations is often challenging. In this thesis, we propose directional splitting techniques and use Newton-type methods that despite these numerical hurdles render reliable and efficient computational schemes. Next, we discuss the problem of choosing the appropriate data fitting term in the case when multiple noise statistics in the data are present due, for instance, to different acquisition and transmission problems. We propose a novel variational model which encodes appropriately and consistently the different noise distributions in this case. Balancing the effect of the regularisation against the data fitting is also crucial. For this sake, we consider a learning approach which estimates the optimal ratio between the two by using training sets of examples via bilevel optimisation. Numerically, we use a combination of SemiSmooth (SSN) and quasi-Newton methods to solve the problem efficiently. Finally, we consider TV-based models in the framework of graphs for image segmentation problems. Here, spectral properties combined with matrix completion techniques are needed to overcome the computational limitations due to the large amount of image data. Further, a semi-supervised technique for the measurement of the segmented region by means of the Hough transform is proposed.