Thesis (Ph. D.)--York University, 2001. Graduate Programme in Mathematics. / Typescript. Includes bibliographical references (leaves 114-119). Also available on the Internet. MODE OF ACCESS via web browser by entering the following URL: http://wwwlib.umi.com/cr/yorku/fullcit?pNQ66358.

In a series of work initiated by Nemirovsky and Yudin, and later extended by Nesterov, first-order algorithms for unconstrained minimization with optimal theoretical complexity bound have been proposed. On the other hand, conjugate gradient algorithms as one of the widely used first-order techniques suffer from the lack of a finite complexity bound. In fact their performance can possibly be quite poor. This dissertation is partially on tightening the gap between these two classes of algorithms, namely the traditional conjugate gradient methods and optimal first-order techniques. We derive conditions under which conjugate gradient methods attain the same complexity bound as in Nemirovsky-Yudin's and Nesterov's methods. Moreover, we propose a conjugate gradient-type algorithm named CGSO, for Conjugate Gradient with Subspace Optimization, achieving the optimal complexity bound with the payoff of a little extra computational cost. We extend the theory of CGSO to convex problems with linear constraints. In particular we focus on solving $l_1$-regularized least square problem, often referred to as Basis Pursuit Denoising (BPDN) problem in the optimization community. BPDN arises in many practical fields including sparse signal recovery, machine learning, and statistics. Solving BPDN is fairly challenging because the size of the involved signals can be quite large; therefore first order methods are of particular interest for these problems. We propose a quasi-Newton proximal method for solving BPDN. Our numerical results suggest that our technique is computationally effective, and can compete favourably with the other state-of-the-art solvers.

The Method of Moments (MoM) has been used extensively to solve electromagnetic problems. Its popularity is largely attributed to its adaptability to structures with various shapes and success in predicting the equivalent induced currents accurately. However, due to its dense matrix, especially for large structures, the MoM suffers from long matrix solution time and large storage requirement. In this thesis it is shown that use of wavelet basis functions result in a MoM matrix which is sparser than the one obtained by using traditional basis functions. A new wavelet system, different from the ones found in literature, is proposed. Stabilized Bi-Conjugate Gradient Method which is an iterative matrix solution method is utilized to solve the resulting sparse matrix equation. Both a one-dimensional problem with a microstrip line example and a two-dimensional problem with a rectangular patch antenna example are studied and the results are compared.

Η συνεισφορά της παρούσας διατριβής επικεντρώνεται στην ανάπτυξη και στη Μαθηματική θεμελίωση νέων μεθόδων συζυγών κλίσεων για βελτιστοποίηση χωρίς περιορισμούς και στη μελέτη νέων μεθόδων εκπαίδευσης νευρωνικών δικτύων και εφαρμογών τους. Αναπτύσσουμε δύο νέες μεθόδους βελτιστοποίησης, οι οποίες ανήκουν στην κλάση των μεθόδων συζυγών κλίσεων. Οι νέες μέθοδοι βασίζονται σε νέες εξισώσεις της τέμνουσας με ισχυρά θεωρητικά πλεονεκτήματα, όπως η προσέγγιση με μεγαλύτερη ακρίβεια της επιφάνεια της αντικειμενικής συνάρτησης. Επιπλέον, μία σημαντική ιδιότητα και των δύο προτεινόμενων μεθόδων είναι ότι εγγυώνται επαρκή μείωση ανεξάρτητα από την ακρίβεια της γραμμικής αναζήτησης, αποφεύγοντας τις συχνά αναποτελεσματικές επανεκκινήσεις. Επίσης, αποδείξαμε την ολική σύγκλιση των προτεινόμενων μεθόδων για μη κυρτές συναρτήσεις. Με βάση τα αριθμητικά μας αποτελέσματα καταλήγουμε στο συμπέρασμα ότι οι νέες μέθοδοι έχουν πολύ καλή υπολογιστική αποτελεσματικότητα, όπως και καλή ταχύτητα επίλυσης των προβλημάτων, υπερτερώντας σημαντικά των κλασικών μεθόδων συζυγών κλίσεων. Το δεύτερο μέρος της διατριβής είναι αφιερωμένο στην ανάπτυξη και στη μελέτη νέων μεθόδων εκπαίδευσης νευρωνικών δικτύων. Προτείνουμε νέες μεθόδους, οι οποίες διατηρούν τα πλεονεκτήματα των κλασικών μεθόδων συζυγών κλίσεων και εξασφαλίζουν τη δημιουργία κατευθύνσεων μείωσης αποφεύγοντας τις συχνά αναποτελεσματικές επανεκκινήσεις. Επιπλέον, αποδείξαμε ότι οι προτεινόμενες μέθοδοι συγκλίνουν ολικά για μη κυρτές συναρτήσεις. Τα αριθμητικά αποτελέσματα επαληθεύουν ότι οι προτεινόμενες μέθοδοι παρέχουν γρήγορη, σταθερότερη και πιο αξιόπιστη σύγκλιση, υπερτερώντας των κλασικών μεθόδων εκπαίδευσης. Η παρουσίαση του ερευνητικού μέρους της διατριβής ολοκληρώνεται με μία νέα μέθοδο εκπαίδευσης νευρωνικών δικτύων, η οποία βασίζεται σε μία καμπυλόγραμμη αναζήτηση. Η μέθοδος χρησιμοποιεί τη BFGS ενημέρωση ελάχιστης μνήμης για τον υπολογισμό των κατευθύνσεων μείωσης, η οποία αντλεί πληροφορία από την ιδιοσύνθεση του προσεγγιστικού Eσσιανού πίνακα, αποφεύγοντας οποιαδήποτε αποθήκευση ή παραγοντοποίηση πίνακα, έτσι ώστε η μέθοδος να μπορεί να εφαρμοστεί για την εκπαίδευση νευρωνικών δικτύων μεγάλης κλίμακας. Ο αλγόριθμος εφαρμόζεται σε προβλήματα από το πεδίο της τεχνητής νοημοσύνης και της βιοπληροφορικής καταγράφοντας πολύ καλά αποτελέσματα. Επίσης, με σκοπό την αύξηση της ικανότητας γενίκευσης των εκπαιδευόμενων δικτύων διερευνήσαμε πειραματικά και αξιολογήσαμε την εφαρμογή τεχνικών μείωσης της διάστασης δεδομένων στην απόδοση της γενίκευσης των τεχνητών νευρωνικών δικτύων σε μεγάλης κλίμακας δεδομένα βιοϊατρικής. / The contribution of this thesis focuses on the development and the Mathematical foundation of new conjugate gradient methods for unconstrained optimization and on the study of new neural network training methods and their applications. We propose two new conjugate gradient methods for unconstrained optimization. The proposed methods are based on new secant equations with strong theoretical advantages i.e. they approximate the surface of the objective function with higher accuracy. Moreover, they have the attractive property of ensuring sufficient descent independent of the accuracy of the line search, avoiding thereby the usual inefficient restarts. Further, we have established the global convergence of the proposed methods for general functions under mild conditions. Based on our numerical results we conclude that our proposed methods outperform classical conjugate gradient methods in both efficiency and robustness. The second part of the thesis is devoted on the study and development of new neural network training algorithms. More specifically, we propose some new training methods which preserve the advantages of classical conjugate gradient methods while simultaneously ensure sufficient descent using any line search, avoiding thereby the usual inefficient restarts. Moreover, we have established the global convergence of our proposed methods for general functions. Encouraging numerical experiments on famous benchmarks verify that the presented methods provide fast, stable and reliable convergence, outperforming classical training methods. Finally, the presentation of the research work of this dissertation is fulfilled with the presentation of a new curvilinear algorithm for training large neural networks which is based on the analysis of the eigenstructure of the memoryless BFGS matrices. The proposed method preserves the strong convergence properties provided by the quasi-Newton direction while simultaneously it exploits the nonconvexity of the error surface through the computation of the negative curvature direction without using any storage and matrix factorization. Our numerical experiments have shown that the proposed method outperforms other popular training methods on famous benchmarks. Furthermore, for improving the generalization capability of trained ANNs, we explore the incorporation of several dimensionality reduction techniques as a pre-processing step. To this end, we have experimentally evaluated the application of dimensional reduction techniques for increasing the generalization capability of neural network in large biomedical datasets.

<p>This dissertation focuses on a FFT based forward EM solver and its application in inverse problems. The main contributions of this work are two folded. On the one hand, it presents the first scaled lab experiment system in the oil and gas industry for through casing hydraulic fracture evaluation. This system is established to validate the feasibility of contrasts enhanced fractures evaluation. On the other hand, this work proposes a FFT based VIE solver for hydraulic fracture evaluation. This efficient solver is needed for numerical analysis of such problem. The solver is then generalized to accommodate scattering simulations for anisotropic inhomogeneous magnetodielectric objects. The inverse problem on anisotropic objects are also studied.</p><p>Before going into details of specific applications, some background knowledge is presented. This dissertation starts with an introduction to inverse problems. Then algorithms for forward and inverse problems are discussed. The discussion on forward problem focuses on the VIE formulation and a frequency domain solver. Discussion on inverse problems focuses on iterative methods.</p><p>The rest of the dissertation is organized by the two categories of inverse problems, namely the inverse source problem and the inverse scattering problem. </p><p>The inverse source problem is studied via an application in microelectronics. In this application, a FFT based inverse source solver is applied to process near field data obtained by near field scanners. Examples show that, with the help of this inverse source solver, the resolution of unknown current source images on a device under test is greatly improved. Due to the improvement in resolution, more flexibility is given to the near field scan system.</p><p>Both the forward and inverse solver for inverse scattering problems are studied in detail. As a forward solver for inverse scattering problems, a fast FFT based method for solving VIE of magnetodielectric objects with large electromagnetic contrasts are presented due to the increasing interest in contrasts enhanced full wave EM imaging. This newly developed VIE solver assigns different basis functions of different orders to expand flux densities and vector potentials. Thus, it is called the mixed ordered BCGS-FFT method. The mixed order BCGS-FFT method maintains benefits of high order basis functions for VIE while keeping correct boundary conditions for flux densities and vector potentials. Examples show that this method has an excellent performance on both isotropic and anisotropic objects with high contrasts. Examples also verify that this method is valid in both high and low frequencies. Based on the mixed order BCGS-FFT method, an inverse scattering solver for anisotropic objects is studied. The inverse solver is formulated and solved by the variational born iterative method. An example given in this section shows a successful inversion on an anisotropic magnetodielectric object. </p><p>Finally, a lab scale hydraulic fractures evaluation system for oil/gas reservoir based on previous discussed inverse solver is presented. This system has been setup to verify the numerical results obtained from previously described inverse solvers. These scaled experiments verify the accuracy of the forward solver as well as the performance of the inverse solver. Examples show that the inverse scattering model is able to evaluate contrasts enhanced hydraulic fractures in a shale formation. Furthermore, this system, for the first time in the oil and gas industry, verifies that hydraulic fractures can be imaged through a metallic casing.</p> / Dissertation

Orientador: Denis José Schiozer / Tese (doutorado) - Universidade Estadual de Campinas, Faculdade de Engenharia Mecânica e Instituto de Geociências / Made available in DSpace on 2018-08-24T00:34:10Z (GMT). No. of bitstreams: 1 Pinto_MarcioAugustoSampaio_D.pdf: 5097853 bytes, checksum: bc8b7f6300987de2beb9a57c26ad806a (MD5) Previous issue date: 2013 / Resumo: Neste trabalho, um método de otimização assistido é proposto para estabelecer uma comparação refinada entre poços convencionais e inteligentes, considerando incertezas geológicas e econômicas. Para isto é apresentada uma metodologia dividida em quatro etapas: (1) representação e operação dos poços no simulador; (2) otimização das camadas/ou blocos completados nos poços convencionais e do número e posicionamento das válvulas nos poços inteligentes; (3) otimização da operação dos poços convencionais e das válvulas nos poços inteligentes, através de um método híbrido de otimização, composto pelo algoritmo genético rápido, para realizar a otimização global, e pelo método de gradiente conjugado, para realizar a otimização local; (4) uma análise de decisão considerando os resultados de todos os cenários geológicos e econômicos. Esta metodologia foi validada em modelos de reservatórios mais simples e com configuração de poços verticais do tipo five-spot, para em seguida ser aplicada em modelos de reservatórios mais complexos, com quatro poços produtores e quatro injetores, todos horizontais. Os resultados mostram uma clara diferença ao aplicar a metodologia proposta para estabelecer a comparação entre os dois tipos de poços. Apresenta também a comparação entre os resultados dos poços inteligentes com três tipos de controle, o reativo e mais duas formas de controle proativo. Os resultados mostram, para os casos utilizados nesta tese, uma ampla vantagem em se utilizar pelo menos uma das formas de controle proativo, ao aumentar a recuperação de óleo e VPL, reduzindo a produção e injeção de água na maioria dos casos / Abstract: In this work, an assisted optimization method is proposed to establish a refined comparison between conventional and intelligent wells, considering geological and economic uncertainties. For this, it is presented a methodology divided into four steps: (1) representation and operation of wells in the simulator, (2) optimization of the layers /blocks with completion in conventional wells and the number and placement of the valves in intelligent wells; (3) optimization of the operation of the conventional and valves in the intelligent, through a hybrid optimization method, comprising by fast genetic algorithm, to perform global optimization, and the conjugate gradient method, to perform local optimization; (4) decision analysis considering the results of all geological and economic scenarios. This method was validated in simple reservoir models and configuration of vertical wells with five-spot type, and then applied to a more complex reservoir model, with four producers and four injectors wells, all horizontal. The results show a clear difference in applying the proposed methodology to establish a comparison between the two types of wells. It also shows the comparison between the results of intelligent wells with three types of control, reactive and two ways of proactive control. The results show, for the cases used in this work, a large advantage to use intelligent wells with at least one form of proactive control, to enhance oil recovery and NPV, reducing water production and injection in most cases / Doutorado / Reservatórios e Gestão / Doutor em Ciências e Engenharia de Petróleo

A parallel realisation of the Conjugate Gradient Method with Schur-Complement preconditioning, based on a domain decomposition approach, is described in detail. Special kinds of solvers for the resulting interiour and coupling systems are presented. A large range of numerical results is used to demonstrate the properties and behaviour of this solvers in practical situations.

This paper concerns the solution of plate bending problems in domains composed of rectangles. Domain decomposition (DD) is the basic tool used for both the parallelization of the conjugate gradient method and the construction of efficient parallel preconditioners. A so-called Dirich- let DD preconditioner for systems of linear equations arising from the fi- nite element approximation by non-conforming Adini elements is derived. It is based on the non-overlapping DD, a multilevel preconditioner for the Schur-complement and a fast, almost direct solution method for the Dirichlet problem in rectangular domains based on fast Fourier transform. Making use of Xu's theory of the auxiliary space method we construct an optimal preconditioner for plate problems discretized by conforming Bogner-Fox-Schmidt rectangles. Results of numerical experiments carried out on a multiprocessor sys- tem are given. For the test problems considered the number of iterations is bounded independent of the mesh sizes and independent of the number of subdomains. The resulting parallel preconditioned conjugate gradient method requiresO(h^-2 ln h^-1 ln epsilon^-11) arithmetical operations per processor in order to solve the finite element equations with the relative accuracy epsilon.

This paper proposes new iterative methods for the efficient computation of the smallest eigenvalue of the symmetric nonlinear matrix eigenvalue problems of large order with a monotone dependence on the spectral parameter. Monotone nonlinear eigenvalue problems for differential equations have important applications in mechanics and physics. The discretization of these eigenvalue problems leads to ill-conditioned nonlinear eigenvalue problems with very large sparse matrices monotone depending on the spectral parameter. To compute the smallest eigenvalue of large matrix nonlinear eigenvalue problem, we suggest preconditioned iterative methods: preconditioned simple iteration method, preconditioned steepest descent method, and preconditioned conjugate gradient method. These methods use only matrix-vector multiplications, preconditioner-vector multiplications, linear operations with vectors and inner products of vectors. We investigate the convergence and derive grid-independent error estimates of these methods for computing eigenvalues. Numerical experiments demonstrate practical effectiveness of the proposed methods for a class of mechanical problems.

Assessing the effect of foam layers on transport phenomena is of significant interest in many industries, so a method for predicting foam layer properties has been developed. A model of the propagation of radiation from an amplitude-modulated laser beam through a non-absorbing foam layer has been developed using diffusion theory. Measurements predicted by diffusion theory were compared to results generated using Monte Carlo methods for a variety of foam layer properties in both the time-domain and the frequency-domain. The properties that were varied include the layer thickness, the scattering coefficient, and the asymmetry parameter. Layer thicknesses between 8.5 mm and 18 cm were considered. Values of the scattering coefficient ranged from about 600 m-1 to 14000 m-1, while the asymmetry parameter varied between 0 and 1. A conjugate-gradient algorithm was used to minimize the difference between simulated Monte Carlo measurements and diffusion theory predicted measurements. A large set of simulated measurements, calculated at various source-detector separations and three discrete frequencies were used to predict the layer properties. Ten blind cases were considered and property predictions were made for each. The predicted properties were within approximately 10% of the actual values, and on average the errors were approximately 4%. Predictions of the reduced scattering coefficient were all within approximately 5% with the majority being within 3%. Predictions of L were all within approximately 10% with the majority being within 7%. Attempts to separate g from the reduced scattering coefficient were unsuccessful, and it was determined that implementation of different source models might make such attempts possible. It was shown that with a large number of measurements, properties could be accurately predicted. A method for reducing the number of measurements needed for accurate property estimation was developed. Starting with a single measurement location, property predictions were made. An approach for updating the optimal detector location, based on the current estimate of the properties, was developed and applied to three cases. Property predictions for the three cases were made to within 10% of the actual values. A maximum of three measurement locations were necessary to obtain such predictions, a significant reduction as compared to the previously illustrated method.