Large scale nonlinear programming (NLP) has proven to be an effective framework for obtaining profit gains through optimal process design and operations in chemical engineering. While the classical SQP and Interior Point methods have been successfully applied to solve many optimization problems, the focus of both academia and industry on larger and more complicated problems requires further development of numerical algorithms which can provide improved computational efficiency. The primary purpose of this dissertation is to develop effective problem formulations and an advanced numerical algorithms for efficient solution of these challenging problems. As problem sizes increase, there is a need for tailored algorithms that can exploit problem specific structure. Furthermore, computer chip manufacturers are no longer focusing on increased clock-speeds, but rather on hyperthreading and multi-core architectures. Therefore, to see continued performance improvement, we must focus on algorithms that can exploit emerging parallel computing architectures. In this dissertation, we develop an advanced parallel solution strategy for nonlinear programming problems with block-angular structure. The effectiveness of this and modern off-the-shelf tools are demonstrated on a wide range of problem classes. Here, we treat optimal design, optimal operation, dynamic optimization, and parameter estimation. Two case studies (air separation units and heat-integrated columns) are investigated to deal with design under uncertainty with rigorous models. For optimal operation, this dissertation takes cryogenic air separation units as a primary case study and focuses on formulations for handling uncertain product demands, contractual constraints on customer satisfaction levels, and variable power pricing. Multiperiod formulations provide operating plans that consider inventory to meet customer demands and improve profits. In the area of dynamic optimization, optimal reference trajectories are determined for load changes in an air separation process. A multiscenario programming formulation is again used, this time with large-scale discretized dynamic models. Finally, to emphasize a different decomposition approach, we address a problem with significant spatial complexity. Unknown water demands within a large scale city-wide distribution network are estimated. This problem provides a different decomposition mechanism than the multiscenario or multiperiod problems; nevertheless, our parallel approach provides effective speedup.

We derive a technique for scaling the search directions of feasible direction methods when applied to optimization problems over Cartesian product sets. It is proved that when the scaling is included in a convergent feasible direction method, also the new method will be convergent. The scaling technique is applied to the Frank-Wolfe method, the partanized Frank-Wolfe method and a heuristic Frank-Wolfe method. The performance of  these algorithms with and without scaling is evaluated on the stochastic transportation problem. It is found that the scaling technique has the ability to improve the performance of some methods. In particular we observed a huge improvement in the performance of the partanized Frank-Wolfe method, especially when the scaling is used together with an exact line search and when the number of sets in the Cartesian product is large.

We consider the problem of reconstructing a surface from noisy samples by approximating the point set with non-uniform rational B-spline surfaces. We focus on the fact that the knot sequences should also be part of the unknown variables that include the control points and the weights in order to find their optimal positions. We show how to set up the free knot problem such that constrained nonlinear optimization can be applied efficiently. We describe in detail a parallel implementation of our approach that give almost linear speedup. Finally, we provide numerical results obtained on the Chemnitzer Linux Cluster supercomputer.

In this dissertation we investigate methods of solving various optimization problems with PDE constraints, i.e. optimization problems that have a system of partial differential equations in the set of constraints, and develop frameworks for a number of practically inspired problems that were not considered in the literature before. Such problems arise in areas like fluid mechanics, chemical engineering, finance, and other areas where a physical system needs to be optimized. In most of the literature sources on PDE-constrained optimization only relatively simple systems of PDEs are considered, they are either linear, or the size of the system is small. On the contrary, in our case, we search for solution methods to problems constrained by large (8 to 10 equations) and non-linear systems of PDEs. More specifically, in the first part of the dissertation we consider a multiphysics phenomenon where electromagnetic and mechanical fields interact within an electrically conductive body, and develop the optimization framework to find an efficient way to control one field through another. We also apply the developed PDE-constrained optimization framework to a financial options portfolio optimization problem, and more specifically consider the case that to the best of our knowledge is not covered in the literature.

Dans cette thèse, nous nous intéressons à la microscopie DIC (Differential interference contrast) en couleur. L’imagerie DIC est reconnue pour produire des images à haut contraste et à haute résolution latérale. L'un de ses inconvénients est que les images observées ne peuvent pas être utilisées directement pour l'interprétation topographique et morphologique, car les changements de phase de la lumière, produits par les variations de l'indice de réfraction de l'objet, sont cachés dans l'image d'intensité. Il s’agit donc d’un problème de reconstruction de phase. Nous présentons un modèle de formation d'image pour la lumière polychromatique, et décrivons de manière détaillée la réponse impulsionnelle du système. Le problème de la reconstruction de phase est abordé sous l’angle d’un problème inverse par minimisation d’un terme d’erreur des moindres carrés (LS) non linéaire avec un terme de régularisation préservant les discontinuités, soit par le potentiel hypersurface (HS), soit par la variation totale (TV). Nous étudions les propriétés des fonctions objectives non convexes résultantes, prouvons l'existence de minimisateurs et proposons une formulation compacte du gradient permettant un calcul rapide. Ensuite, nous proposons des outils d'optimisation efficaces récents permettant d'obtenir à la fois des reconstructions précises pour les deux régularisations lisse (HS) et non lisse (TV) et des temps de calculs réduits. / In this dissertation we address the problem of estimating the phase from colorimages acquired with differential–interference–contrast (DIC) microscopy. This technique has been widely recognized for producing high contrast images at high lateral resolution. One of its disadvant ages is that the observed images cannot be easily used for topographical and morphological interpretation, because the changes in phase of the light, produced by variations in the refractive index of the object, are hidden in the intensity image. We present an image formation model for polychromatic light, along with a detailed description of the point spread function (PSF). As for the phase recovery problem, we followed the inverse problem approach by means of minimizing a non-linear least–squares (LS)–like discrepancy term with an edge–preserving regularizing term, given by either the hypersurface (HS) potential or the total variation (TV) one. We investigate the analytical properties of the resulting objective non-convex functions, prove the existence of minimizers and propose a compact formulation of the gradient allowing fast computations. Then we use recent effective optimization tools able to obtain in both the smooth and the non-smooth cases accurate reconstructions with a reduced computational demand. We performed different numerical tests on synthetic realistic images and we compared the proposed methods with both the original conjugate gradient method proposed in the literature, exploiting a gradient–free linesearch for the computation of the steplength parameter, and other standard conjugate gradient approaches.

This paper is concerned with approximating noisy samples by non-uniform rational B-spline curves with special emphasis on free knots. We show how to set up the problem such that nonlinear optimization methods can be applied efficiently. This involves the introduction of penalizing terms in order to avoid undesired knot positions. We report on our implementation of the nonlinear optimization and we show a way to implement the program in parallel. Parallel performance results are described. Our experiments show that our program has a linear speedup and an efficiency value close to unity. Runtime results on a parallel computer are displayed.

Electromagnetic imaging is the problem of determining material properties from scattered fields measured away from the domain under investigation. Solving this inverse problem is a challenging task because (i) it is ill-posed due to the presence of (smoothing) integral operators used in the representation of scattered fields in terms of material properties, and scattered fields are obtained at a finite set of points through noisy measurements; and (ii) it is nonlinear simply due the fact that scattered fields are nonlinear functions of the material properties. The work described in this thesis tackles the ill-posedness of the electromagnetic imaging problem using sparsity-based regularization techniques, which assume that the scatterer(s) occupy only a small fraction of the investigation domain. More specifically, four novel imaging methods are formulated and implemented. (i) Sparsity-regularized Born iterative method iteratively linearizes the nonlinear inverse scattering problem and each linear problem is regularized using an improved iterative shrinkage algorithm enforcing the sparsity constraint. (ii) Sparsity-regularized nonlinear inexact Newton method calls for the solution of a linear system involving the Frechet derivative matrix of the forward scattering operator at every iteration step. For faster convergence, the solution of this matrix system is regularized under the sparsity constraint and preconditioned by leveling the matrix singular values. (iii) Sparsity-regularized nonlinear Tikhonov method directly solves the nonlinear minimization problem using Landweber iterations, where a thresholding function is applied at every iteration step to enforce the sparsity constraint. (iv) This last scheme is accelerated using a projected steepest descent method when it is applied to three-dimensional investigation domains. Projection replaces the thresholding operation and enforces the sparsity constraint. Numerical experiments, which are carried out using synthetically generated or actually measured scattered fields, show that the images recovered by these sparsity-regularized methods are sharper and more accurate than those produced by existing methods. The methods developed in this work have potential application areas ranging from oil/gas reservoir engineering to biological imaging where sparse domains naturally exist.

This dissertation is concerned with the unconstrained global optimization of nonlinear problems. These problems are not easy to solve because of the multiplicity of local and global minima. In this dissertation, we first study the pattern search method for local optimization. We study the pattern search method numerically and provide a modification to it. In particular, we design a new pattern search method for local optimization. The new pattern search improves the efficiency and reliability of the original pattern search method. We then designed two simulated annealing algorithms for global optimization based on the basic features of pattern search. The new methods are therefore hybrid. The first hybrid method is the hybrid of simulated annealing and pattern search. This method is denoted by MSA. The second hybrid method is a combination of MSA and the multi-level single linkage method. This method is denoted by SAPS. The performance of MSA and SAPS are reported through extensive experiments on 50 test problems. Results indicate that the new hybrids are efficient and reliable.

For a numerical or analytical model to be useful it should be ensured that the model outcome matches the observations or field measurements during calibration. This process has been typically done by manual perturbation of the model input parameters. This research investigates a methodology for using non linear parameter estimation technique (the Marquardt-Levenberg technique) with the multi component reactive solute transport model SEAM3D. The reactive multi-component solutes considered in this study are chlorinated ethenes. Previous studies have shown that this class of compounds can be degraded by four different biodegradation mechanisms, and the degradation path is a function of the prevailing oxidation reduction conditions. Tests were performed in three levels; the first level utilized synthetic model-generated data. The idea was to develop a methodology and perform preliminary testing where "observations" can be generated as needed. The second level of testing involved performing the testing on a single redox zone model. The methodology was refined and tested using data from a chlorinated ethenes-contaminated site. The third level involved performing the tests on a multiple redox zone model. The methodology was tested, and statistical validation of the recommended methodology was performed. The results of the tests showed that there is a statistical advantage for choosing a subgroup of the available parameters to optimize instead of the optimizing the whole available group. Therefore, it is recommended to perform a parameter sensitivity study prior to the optimization process to identify the suitable parameters to be chosen. The methodology suggests optimizing the oxidation-reduction species parameters first then calibrating the chlorinated ethenes model. The results of the tests also proved the advantage of the sequential optimization of the model parameters, therefore the parameters of the parent compound are optimized, updated in the daughter compound model, for which the parameters are then optimized so on. The test results suggested considering the concentrations of the daughter compounds when optimizing the parameters of the parent compounds. As for the observation weights, the tests suggest starting the applied observation weights during the optimization process at values of one and changing them if needed. Overall the proposed methodology proved to be very efficient. The optimization methodology yielded sets of model parameters capable of generating concentration profiles with great resemblance to the observed concentration profiles in the two chlorinated ethenes site models considered. / Ph. D.

Since augmented Lagrangian methods were introduced by Powell and Hestenes, this class of methods has been investigated very intensively. While the finite dimensional case has been treated in a satisfactory manner, the infinite dimensional case is studied much less. The general approach to solve an infinite dimensional optimization problem subject to equality constraints is as follows: First one proves convergence for a basic algorithm in the Hilbert space setting. Then one discretizes the given spaces and operators in order to make numerical computations possible. Finally, one constructs a discretized version of the infinite dimensional method and tries to transfer the convergence results to the finite dimensional version of the basic algorithm. In this thesis we discuss a globally convergent augmented Lagrangian algorithm and discretize it in terms of functional analytic restriction operators. Given this setting, we prove global convergence of the discretized version of this algorithm to a stationary point of the infinite dimensional optimization problem. The proposed algorithm includes an explicit rule of how to update the discretization level and the penalty parameter from one iteration to the next one - questions that had been unanswered so far. In particular the latter update rule guarantees that the penalty parameters stay bounded away from zero which prevents the Hessian of the discretized augmented Lagrangian functional from becoming more and more ill conditioned. / Master of Science