Thesis (M. Eng.)(Electronics)--University of Pretoria, 2001. / Title from opening screen (viewed March 10, 2005. Summaries in Afrikaans and English. Includes bibliography and index.

Thesis (Ph. D.)--Industrial and Systems Engineering, Georgia Institute of Technology, 2006. / Parker, R. Gary, Committee Member ; Shapiro, Alexander, Committee Member ; Nemirovski, Arkadi, Committee Member ; Green, William, Committee Member ; Monteiro, Renato, Committee Chair.

Orientadores: Aurélio Ribeiro Leite de Oliveira, Marta Ines Velazco Fontova / Dissertação (mestrado) - Universidade Estadual de Campinas, Instituto de Matemática, Estatística e Computação Científica / Made available in DSpace on 2018-08-18T12:49:39Z (GMT). No. of bitstreams: 1 Coelho_AlessandroFonsecaEsteves_M.pdf: 2659631 bytes, checksum: fc1bec925179612ee07a4aaef7092d8a (MD5) Previous issue date: 2011 / Resumo: Um método frequentemente utilizado para a solução de problemas de programação linear é o método de pontos interiores. Nestes métodos precisamos resolver sistemas lineares para calcular a direção de Newton a cada iteração. A solução desses sistemas consiste no passo de maior esforço computacional nos métodos de pontos interiores. A fatoração de Cholesky é a opção mais utilizada para resolver estes sistemas. Contudo, quando trabalhamos com problemas de grande porte, esta fatoração pode ser densa e torna-se inviável trabalhar com esses métodos. Nestes casos, uma boa opção consiste no uso de métodos iterativos precondicionados. Estudos anteriores utilizam o método dos gradientes conjugados precondicionado para obter uma solução destes sistemas. Particularmente, os sistemas originados dos métodos de pontos interiores, são, naturalmente, sistemas de equações normais. Porém, a versão padrão do método dos gradientes conjugados, não considera a estrutura de equações normais do sistema. Neste trabalho propomos a utilização de duas versões do método de gradientes conjugados precondicionado que consideram a estrutura de equações normais destes sistemas. Estas versões serão comparadas com a versão de gradientes conjugados precondicionada que não considera a estrutura de equações normais do sistema. Resultados numéricos com problemas de grande porte mostram que uma dessas versões é competitiva em relação à versão padrão / Abstract: An often used method for solving linear programming problems is the interior point method. In these methods we need to solve linear systems to compute the Newton search direction at each iteration. The solution of these systems is the procedure of most computational effort in interior point methods. The Cholesky factorization is the most often used method to solve these systems. However, when dealing with large scale problems, this factorization can be dense and it become impossible to apply such methods. In such cases, a good option is the use of preconditioned iterative methods. Previous studies have used the preconditioned conjugate gradient method to find the solution of these systems. Particularly, the systems arising from interior point methods are, naturally, systems of normal equations type. Nevertheless, the standard version of the conjugate gradient method, does not take into account the normal equations system structure. This study proposes the use of two versions of preconditioned conjugate gradient method considering the normal equations structure of these systems. These versions are compared with the preconditioned conjugate gradient version that does not consider that structure. Numerical results with large scale problems show that one of these versions is competitive with the standard one / Mestrado / Matematica Aplicada / Mestre em Matemática Aplicada

Thesis (Ph. D.)--York University, 2001. Graduate Programme in Mathematics. / Typescript. Includes bibliographical references (leaves 114-119). Also available on the Internet. MODE OF ACCESS via web browser by entering the following URL: http://wwwlib.umi.com/cr/yorku/fullcit?pNQ66358.

Η συνεισφορά της παρούσας διατριβής επικεντρώνεται στην ανάπτυξη και στη Μαθηματική θεμελίωση νέων μεθόδων συζυγών κλίσεων για βελτιστοποίηση χωρίς περιορισμούς και στη μελέτη νέων μεθόδων εκπαίδευσης νευρωνικών δικτύων και εφαρμογών τους. Αναπτύσσουμε δύο νέες μεθόδους βελτιστοποίησης, οι οποίες ανήκουν στην κλάση των μεθόδων συζυγών κλίσεων. Οι νέες μέθοδοι βασίζονται σε νέες εξισώσεις της τέμνουσας με ισχυρά θεωρητικά πλεονεκτήματα, όπως η προσέγγιση με μεγαλύτερη ακρίβεια της επιφάνεια της αντικειμενικής συνάρτησης. Επιπλέον, μία σημαντική ιδιότητα και των δύο προτεινόμενων μεθόδων είναι ότι εγγυώνται επαρκή μείωση ανεξάρτητα από την ακρίβεια της γραμμικής αναζήτησης, αποφεύγοντας τις συχνά αναποτελεσματικές επανεκκινήσεις. Επίσης, αποδείξαμε την ολική σύγκλιση των προτεινόμενων μεθόδων για μη κυρτές συναρτήσεις. Με βάση τα αριθμητικά μας αποτελέσματα καταλήγουμε στο συμπέρασμα ότι οι νέες μέθοδοι έχουν πολύ καλή υπολογιστική αποτελεσματικότητα, όπως και καλή ταχύτητα επίλυσης των προβλημάτων, υπερτερώντας σημαντικά των κλασικών μεθόδων συζυγών κλίσεων. Το δεύτερο μέρος της διατριβής είναι αφιερωμένο στην ανάπτυξη και στη μελέτη νέων μεθόδων εκπαίδευσης νευρωνικών δικτύων. Προτείνουμε νέες μεθόδους, οι οποίες διατηρούν τα πλεονεκτήματα των κλασικών μεθόδων συζυγών κλίσεων και εξασφαλίζουν τη δημιουργία κατευθύνσεων μείωσης αποφεύγοντας τις συχνά αναποτελεσματικές επανεκκινήσεις. Επιπλέον, αποδείξαμε ότι οι προτεινόμενες μέθοδοι συγκλίνουν ολικά για μη κυρτές συναρτήσεις. Τα αριθμητικά αποτελέσματα επαληθεύουν ότι οι προτεινόμενες μέθοδοι παρέχουν γρήγορη, σταθερότερη και πιο αξιόπιστη σύγκλιση, υπερτερώντας των κλασικών μεθόδων εκπαίδευσης. Η παρουσίαση του ερευνητικού μέρους της διατριβής ολοκληρώνεται με μία νέα μέθοδο εκπαίδευσης νευρωνικών δικτύων, η οποία βασίζεται σε μία καμπυλόγραμμη αναζήτηση. Η μέθοδος χρησιμοποιεί τη BFGS ενημέρωση ελάχιστης μνήμης για τον υπολογισμό των κατευθύνσεων μείωσης, η οποία αντλεί πληροφορία από την ιδιοσύνθεση του προσεγγιστικού Eσσιανού πίνακα, αποφεύγοντας οποιαδήποτε αποθήκευση ή παραγοντοποίηση πίνακα, έτσι ώστε η μέθοδος να μπορεί να εφαρμοστεί για την εκπαίδευση νευρωνικών δικτύων μεγάλης κλίμακας. Ο αλγόριθμος εφαρμόζεται σε προβλήματα από το πεδίο της τεχνητής νοημοσύνης και της βιοπληροφορικής καταγράφοντας πολύ καλά αποτελέσματα. Επίσης, με σκοπό την αύξηση της ικανότητας γενίκευσης των εκπαιδευόμενων δικτύων διερευνήσαμε πειραματικά και αξιολογήσαμε την εφαρμογή τεχνικών μείωσης της διάστασης δεδομένων στην απόδοση της γενίκευσης των τεχνητών νευρωνικών δικτύων σε μεγάλης κλίμακας δεδομένα βιοϊατρικής. / The contribution of this thesis focuses on the development and the Mathematical foundation of new conjugate gradient methods for unconstrained optimization and on the study of new neural network training methods and their applications. We propose two new conjugate gradient methods for unconstrained optimization. The proposed methods are based on new secant equations with strong theoretical advantages i.e. they approximate the surface of the objective function with higher accuracy. Moreover, they have the attractive property of ensuring sufficient descent independent of the accuracy of the line search, avoiding thereby the usual inefficient restarts. Further, we have established the global convergence of the proposed methods for general functions under mild conditions. Based on our numerical results we conclude that our proposed methods outperform classical conjugate gradient methods in both efficiency and robustness. The second part of the thesis is devoted on the study and development of new neural network training algorithms. More specifically, we propose some new training methods which preserve the advantages of classical conjugate gradient methods while simultaneously ensure sufficient descent using any line search, avoiding thereby the usual inefficient restarts. Moreover, we have established the global convergence of our proposed methods for general functions. Encouraging numerical experiments on famous benchmarks verify that the presented methods provide fast, stable and reliable convergence, outperforming classical training methods. Finally, the presentation of the research work of this dissertation is fulfilled with the presentation of a new curvilinear algorithm for training large neural networks which is based on the analysis of the eigenstructure of the memoryless BFGS matrices. The proposed method preserves the strong convergence properties provided by the quasi-Newton direction while simultaneously it exploits the nonconvexity of the error surface through the computation of the negative curvature direction without using any storage and matrix factorization. Our numerical experiments have shown that the proposed method outperforms other popular training methods on famous benchmarks. Furthermore, for improving the generalization capability of trained ANNs, we explore the incorporation of several dimensionality reduction techniques as a pre-processing step. To this end, we have experimentally evaluated the application of dimensional reduction techniques for increasing the generalization capability of neural network in large biomedical datasets.

Orientador: Denis José Schiozer / Tese (doutorado) - Universidade Estadual de Campinas, Faculdade de Engenharia Mecânica e Instituto de Geociências / Made available in DSpace on 2018-08-24T00:34:10Z (GMT). No. of bitstreams: 1 Pinto_MarcioAugustoSampaio_D.pdf: 5097853 bytes, checksum: bc8b7f6300987de2beb9a57c26ad806a (MD5) Previous issue date: 2013 / Resumo: Neste trabalho, um método de otimização assistido é proposto para estabelecer uma comparação refinada entre poços convencionais e inteligentes, considerando incertezas geológicas e econômicas. Para isto é apresentada uma metodologia dividida em quatro etapas: (1) representação e operação dos poços no simulador; (2) otimização das camadas/ou blocos completados nos poços convencionais e do número e posicionamento das válvulas nos poços inteligentes; (3) otimização da operação dos poços convencionais e das válvulas nos poços inteligentes, através de um método híbrido de otimização, composto pelo algoritmo genético rápido, para realizar a otimização global, e pelo método de gradiente conjugado, para realizar a otimização local; (4) uma análise de decisão considerando os resultados de todos os cenários geológicos e econômicos. Esta metodologia foi validada em modelos de reservatórios mais simples e com configuração de poços verticais do tipo five-spot, para em seguida ser aplicada em modelos de reservatórios mais complexos, com quatro poços produtores e quatro injetores, todos horizontais. Os resultados mostram uma clara diferença ao aplicar a metodologia proposta para estabelecer a comparação entre os dois tipos de poços. Apresenta também a comparação entre os resultados dos poços inteligentes com três tipos de controle, o reativo e mais duas formas de controle proativo. Os resultados mostram, para os casos utilizados nesta tese, uma ampla vantagem em se utilizar pelo menos uma das formas de controle proativo, ao aumentar a recuperação de óleo e VPL, reduzindo a produção e injeção de água na maioria dos casos / Abstract: In this work, an assisted optimization method is proposed to establish a refined comparison between conventional and intelligent wells, considering geological and economic uncertainties. For this, it is presented a methodology divided into four steps: (1) representation and operation of wells in the simulator, (2) optimization of the layers /blocks with completion in conventional wells and the number and placement of the valves in intelligent wells; (3) optimization of the operation of the conventional and valves in the intelligent, through a hybrid optimization method, comprising by fast genetic algorithm, to perform global optimization, and the conjugate gradient method, to perform local optimization; (4) decision analysis considering the results of all geological and economic scenarios. This method was validated in simple reservoir models and configuration of vertical wells with five-spot type, and then applied to a more complex reservoir model, with four producers and four injectors wells, all horizontal. The results show a clear difference in applying the proposed methodology to establish a comparison between the two types of wells. It also shows the comparison between the results of intelligent wells with three types of control, reactive and two ways of proactive control. The results show, for the cases used in this work, a large advantage to use intelligent wells with at least one form of proactive control, to enhance oil recovery and NPV, reducing water production and injection in most cases / Doutorado / Reservatórios e Gestão / Doutor em Ciências e Engenharia de Petróleo

Over the last 25 years, interior-point methods (IPMs) have emerged as a viable class of algorithms for solving various forms of conic optimization problems. Most IPMs use a modified Newton method to determine the search direction at each iteration. The system of equations corresponding to the modified Newton system can often be reduced to the so-called normal equation, a system of equations whose matrix ADA' is positive definite, yet often ill-conditioned. In this thesis, we first investigate the theoretical properties of the maximum weight basis (MWB) preconditioner, and show that when applied to a matrix of the form ADA', where D is positive definite and diagonal, the MWB preconditioner yields a preconditioned matrix whose condition number is uniformly bounded by a constant depending only on A. Next, we incorporate the results regarding the MWB preconditioner into infeasible, long-step, primal-dual, path-following algorithms for linear programming (LP) and convex quadratic programming (CQP). In both LP and CQP, we show that the number of iterative solver iterations of the algorithms can be uniformly bounded by n and a condition number of A, while the algorithmic iterations of the IPMs can be polynomially bounded by n and the logarithm of the desired accuracy. We also expand the scope of the LP and CQP algorithms to incorporate a family of preconditioners, of which MWB is a member, to determine an approximate solution to the normal equation. For the remainder of the thesis, we develop a new preconditioning strategy for solving systems of equations whose associated matrix is positive definite but ill-conditioned. Our so-called adaptive preconditioning strategy allows one to change the preconditioner during the course of the conjugate gradient (CG) algorithm by post-multiplying the current preconditioner by a simple matrix, consisting of the identity matrix plus a rank-one update. Our resulting algorithm, the Adaptive Preconditioned CG (APCG) algorithm, is shown to have polynomial convergence properties. Numerical tests are conducted to compare a variant of the APCG algorithm with the CG algorithm on various matrices.

Este trabalho apresenta um método de reconstrução de imagens de ultrassom por problemas inversos que tem como penalidade para o erro entre solução e dados a norma L2, ou euclidiana, e como penalidade de regularização a norma L1. A motivação para o uso da regularização L1 é que se trata de um tipo de regularização promotora de esparsidade na solução. A esparsidade da regularização L1 contorna o problema de excesso do artefatos, observado em outras implementações de reconstrução por problemas inversos em ultrassom. Este problema é consequência principalmente da limitação da representação discreta do objeto contínuo no modelo de aquisição. Por conta desta limitação, objetos refletores na área imageada quase sempre localizam-se em posições que não correspondem precisamente a uma das posições do modelo discreto, gerando dados que não correspondem aos dados modelados. As formulações do problema com regularização L2 e com regularização L1 são apresentadas e comparadas dos pontos de vista geométrico e Bayesiano. O algoritmo de otimização proposto é uma implementação do algoritmo Iteratively Reweighted Least Squares (IRLS) e utiliza o método do Gradiente Conjugado (CG - Conjugate Gradient) a cada iteração, sendo chamado de IRLS-CG. São realizadas simulações com phantoms computacionais que mostram que o método permite reconstruir imagens a partir da aquisição de dados com refletores em posições não modeladas sem a observação de artefatos. As simulações também mostram melhor resolução espacial do método proposto com relação ao algoritmo delay-and-sum (DAS). Também se observou melhor desempenho computacional do CG com relação à matriz inversa nas iterações do IRLS. / This work presents an inverse problem based method for ultrasound image reconstruction which uses the L2-norm (or euclidean norm) as a penalty for the error between the data and the solution, and the L1-norm as a regularization penalty. The motivation for the use of of L1 regularization is the sparsity promoting property of this type of regularization. The sparsity of L1 regularization circumvents the problem of excess of artifatcts that is observed in other approaches of inverse problem based reconstrucion in ultrasound. Such problem is mainly a consequence of the limitation in the discrete representation of a continuous object in the acquisition model. Due to this limitation, reflecting objects in the imaged area are often localized in positions that do not correspond precisely to one of the positions in the discrete model, therefore generating data that do not correspond to the model data. The formulations of the problem with L2 regularization and with L1 regularization are presented and compared in geometric and Bayesian terms. The optimization algorithm proposed is an implementation of Iteratively Reweighted Least Squares (IRLS) and uses the Conjugate Gradient (CG) method inside each iteration, thus being called IRLS-CG. Simulations with computer phantoms are realized showing that the proposed method allows for the reconstruction of images, without observable artifacts, from data with reflectors located in non-modeled positions. Simulations also show a better spatial resolution in the proposed method when compared to the delay-and-sum (DAS) algorithm. It was also observed better computational performance of CG when compared to the matrix inversion in the iterations of IRLS.

Este trabalho apresenta um método de reconstrução de imagens de ultrassom por problemas inversos que tem como penalidade para o erro entre solução e dados a norma L2, ou euclidiana, e como penalidade de regularização a norma L1. A motivação para o uso da regularização L1 é que se trata de um tipo de regularização promotora de esparsidade na solução. A esparsidade da regularização L1 contorna o problema de excesso do artefatos, observado em outras implementações de reconstrução por problemas inversos em ultrassom. Este problema é consequência principalmente da limitação da representação discreta do objeto contínuo no modelo de aquisição. Por conta desta limitação, objetos refletores na área imageada quase sempre localizam-se em posições que não correspondem precisamente a uma das posições do modelo discreto, gerando dados que não correspondem aos dados modelados. As formulações do problema com regularização L2 e com regularização L1 são apresentadas e comparadas dos pontos de vista geométrico e Bayesiano. O algoritmo de otimização proposto é uma implementação do algoritmo Iteratively Reweighted Least Squares (IRLS) e utiliza o método do Gradiente Conjugado (CG - Conjugate Gradient) a cada iteração, sendo chamado de IRLS-CG. São realizadas simulações com phantoms computacionais que mostram que o método permite reconstruir imagens a partir da aquisição de dados com refletores em posições não modeladas sem a observação de artefatos. As simulações também mostram melhor resolução espacial do método proposto com relação ao algoritmo delay-and-sum (DAS). Também se observou melhor desempenho computacional do CG com relação à matriz inversa nas iterações do IRLS. / This work presents an inverse problem based method for ultrasound image reconstruction which uses the L2-norm (or euclidean norm) as a penalty for the error between the data and the solution, and the L1-norm as a regularization penalty. The motivation for the use of of L1 regularization is the sparsity promoting property of this type of regularization. The sparsity of L1 regularization circumvents the problem of excess of artifatcts that is observed in other approaches of inverse problem based reconstrucion in ultrasound. Such problem is mainly a consequence of the limitation in the discrete representation of a continuous object in the acquisition model. Due to this limitation, reflecting objects in the imaged area are often localized in positions that do not correspond precisely to one of the positions in the discrete model, therefore generating data that do not correspond to the model data. The formulations of the problem with L2 regularization and with L1 regularization are presented and compared in geometric and Bayesian terms. The optimization algorithm proposed is an implementation of Iteratively Reweighted Least Squares (IRLS) and uses the Conjugate Gradient (CG) method inside each iteration, thus being called IRLS-CG. Simulations with computer phantoms are realized showing that the proposed method allows for the reconstruction of images, without observable artifacts, from data with reflectors located in non-modeled positions. Simulations also show a better spatial resolution in the proposed method when compared to the delay-and-sum (DAS) algorithm. It was also observed better computational performance of CG when compared to the matrix inversion in the iterations of IRLS.

Orientador: Eduardo Cardoso de Abreu / Dissertação (mestrado) - Universidade Estadual de Campinas, Instituto de Matemática Estatística e Computação Científica / Made available in DSpace on 2018-08-26T22:28:13Z (GMT). No. of bitstreams: 1 Ferraz_PaolaCunha_M.pdf: 6535346 bytes, checksum: 5f9c9ba53cd3e63fc60c09c90ad2c625 (MD5) Previous issue date: 2015 / Resumo: O foco deste trabalho é aproximação numérica de problemas envolvendo equações diferenciais parciais (EDPs), de natureza elíptica, no contexto de aplicações em dinâmica de fluidos em meios porosos. Especificamente, a dissertação pretende contribuir com uma implementação de um algoritmo multiescala e multigrid, recentemente introduzido na literatura, para resolução aproximada de sistemas de equações lineares de grande porte e mal condicionados, proveniente dessa classe de EDPs, tipicamente associada a problemas de Poisson de pressão-velocidade com condições de contornos típicas de fluxo em meios porosos. O problema concreto de Poisson discutido neste trabalho será desacoplado do sistema de transporte de EDPs de convecção-difusão, com convecção dominante, e linearizado por meio do emprego de uma técnica de decomposição de operadores. A metodologia para a discretização do problema elíptico de Poisson é elementos finitos mistos híbridos. A resolução numérica do sistema linear resultante deste procedimento será realizado via um método do tipo Gradientes Conjugados com Pré-condicionamento (PCG) multiescala e multigrid. Combinamos as metodologias multi-escala e multigrid de modo a capturar os distintos comprimentos de onda associados aos diferentes comprimentos de onda do operador diferencial auto-adjunto de Poisson, fortemente influenciado pela heterogeneidade das propriedades geológicas do meio poroso, em particular da permeabilidade absoluta, que pode exibir flutuações em várias ordens de grandeza. Experimentos computacionais em aplicações de problemas de dinâmica de fluidos em meios porosos são apresentados e discutidos para verificação dos resultados obtidos / Abstract: The focus of this work is the numerical approximation of differential problems involving partial differential equations (PDE's) of elliptic nature, in the context of modelling and simulation in fluid dynamics in porous media. The dissertation aims to contribute with an implementation of a multiscale multigrid algorithm, recently introduced in the literature, designed for solving ill-conditioned large linear systems of equations derived from those classes of PDE's, typically associated with Poisson problems of pressure-velocity with boundary conditions typical of flow in porous media. The Poisson problem discussed here is identified from the coupled convection-diffusion transport system counterpart of PDE's, with dominated convection, and by a linearization by means the use of an operator splitting approach. The methodology used for the discretization of the Poisson elliptic problem is by mixed hybrid finite elements. The numerical solution of the resulting linear system will be addressed by a multiscale multigrid preconditioned conjugate gradient (PCG) method. We combine both methodologies in order to capture the distinct wavelengths associated with the different wavelengths from the assosiated self-adjoint Poisson operator, strongly influenced by the heterogeneity of the geological properties of the porous media, in particular to the absolute permeability tensor, which in turn might exhibit very large fluctuations of orders of magnitude. Numerical experiments in applications of fluid dynamics problems in porous media are presented and discussed for a verification of the results obtained by direct numerical simulations with the multiscale multigrid algorithm under consideration / Mestrado / Matematica Aplicada / Mestra em Matemática Aplicada